Do any decision problems exist outside NP and NP-Hard?

Question

This question asks about NP-hard problems that are not NP-complete. I'm wondering if there exist any decision problems that are neither NP nor NP-hard.

In order to be in NP, problems have to have a verifier that runs in polynomial time on a deterministic Turing machine. Obviously, all problems in P meet that criteria, but what about the problems with sub-exponential complexity? They do not belong to P and it's not obvious to me that they all have efficient deciders. And they certainly don't qualify for NP-complete.

I'm willing to believe that all decision problems are either NP or NP-hard or both, but nobody has actually said that (that I can find). I'm also willing to believe that such problems do exist, even if they are very contrived. Maybe someone more knowledgeable can put this issue to rest for me. Thanks.

Edit

I abused the term 'subexponential' in my question. In my mind it meant some problem with a complexity between exponential and polynomial like L-notation in this table. See the links in Raphael's answer for more details.

Answer 1

If $P = NP$, then any non-trivial language is NP-hard, and any trivial language belongs to NP. Hence, we do not get anything which is neither NP or NP-hard in this case.

If, however, $P \neq NP$, then there are languages which are neither in NP nor NP-hard.

For example, we can consider the language $\{1^n \mid \text{the } n\text{-th TM halts}\}$. As this language is not computable, it is clearly outside of $NP$. As this language is sparse, $P \neq NP$ together with Mahaney's theorem implies that it is not NP-hard.

If we want a computable counterexample, we can take a language $L$ that is not in EXPSPACE. We number all finite words in increasing order as $(w_n)_{n \in \mathbb{N}}$, and now consider the language $L' = \{1^n \mid w_n \in L\}$. Again, Mahaney's theorem ensures that $L'$ is not NP-hard. If $L'$ were in NP (or even PSPACE), then an EXPSPACE algorithm for $L$ would be available in "Given w_n, compute 1^n and run the PSPACE-algorithm for $L'$ on it."

See here https://math.stackexchange.com/questions/235162/if-an-unary-language-exists-in-npc-then-p-np for a proof of Mahaney's theorem. Note that Yuval speaks about starting with an NP-complete unary language, that assumption is not used - the argument only needs an NP-hard unary language.

Answer 2

Assuming $P\not=NP$ there are lots of problems incomparable with $SAT$, hence neither $NP$ nor $NP$-hard.

Here's an overkill generalization of this fact:

Suppose $X\not\in P$. Then there is some $Y$ such that $Y$ is incomparable with $X$ under polynomial-time-Turing reducibility (hence a fortiori Karp reducibility).

(So in particular if $P\not=NP$ we get an affirmative answer to the OP.)

One way to prove this is with forcing. Fix an $X\not\in P$ and let $Y$ be sufficiently generic with respect to $X$ ($2$-generic with respect to $X$ is already overkill). Since only countably many things are reducible to $X$, a sufficient level of genericity will ensure that $Y$ is not reducible to $X$. So we only have to treat the possibility that $Y\ge_pX$.

By genericity, if $Y\ge_pX$ there is some condition $\sigma$ (= "finite initial segment" of $Y$, thinking of languages as infinite binary strings) such that for some number $e$ and polynomial $p$, $\sigma$ forces "$\Phi_e$ is a $p$-bounded Turing reduction of $X$ to $Y$." But this lets us compute $X$ in polynomial time as follows: to tell whether $n\in Y$, we just check whether $$\Phi_e^{\sigma^\smallfrown 1^{p(\vert n\vert)}}(n)\downarrow=1,$$ which can be done in polynomial time.

EDIT: note that this argument produces non-decidable languages since "sufficiently generic" languages are not decidable. This is both good and bad: it shows that in a strong way that "hard to compute" and "computes a lot" aren't really the same, but it also means that the examples so produced aren't of general complexity-theoretic interest. Constructing decidable examples takes a finer-grained argument, and Arno's answer does precisely this. In general forcing arguments can often prove very broad results in quite snappy ways, but usually don't result in "nice" objects. The original example of this is probably the fact that forcing can build incomparable $\Delta^0_2$ Turing degrees but we need priority arguments to build incomparable c.e. Turing degrees, but Baker-Gill-Solovay provides another good example since there is - to my knowledge - no known "nice" oracle relative to which we have $P=NP$, despite the existence of such being not too hard to prove via forcing.

---

The proof above immediately yields a strengthening of the claim: for any $X$, the set of sufficiently generic $Y$s is always comeager, so in a precise sense "most" languages will be incomparable with $X$ as long as $X\not\in P$.

(I strongly suspect the same is true for measure as well, but I don't have an argument at the moment.)