To expand a little on Luke's answer, physically building a neural net to solve any language requires producing electronic components with infinitely precise resistances and so on. This isn't possible, in multiple ways:
You can't produce a resistor of exactly $\mathrm{2\,\Omega}$.
Resistance changes with temperature and current flowing through the resistor will change its temperature.
Even supposing that you know an electronic engineer/sorcerer who can produce resistors to any exact value you choose and that don't change resistance with temperature, setting up your machine to decide an uncomputable language will require uncomputable resistance values. So there's no way you could actually tell your electronic engineer/sorcerer what resistance value you need.
So, although, in principle, these machines can decide any language, they don't violate Church–Turing because they can't be physically constructed.
You might want to engage in some rules-lawyering and claim that somebody could hand you one of these machines and say, "Hey, look, this machine just happens to have exactly the right resistance values to solve the halting problem!" However, they have no way of proving this claim, since they can't measure the components to infinite precision, so the best they can claim with justification is "I tested this on some finite set of inputs and it correctly decided the halting problem on those inputs." Well, any finite subset of the halting problem is already Turing-decidable so that's nothing exciting.