C++ like performance?

Current License: CC BY-SA 3.0

34 events

when toggle format	what		by	license	comment
Apr 9, 2012 at 19:41	comment	added	Robert Cartaino		Notice: This comment-thread is developing into a mini chat room. If you'd like to continue this discussion, please bring it to chat. Comments should be used to improve the original post. Please discontinue this conversation here. Thanks.
Apr 2, 2012 at 9:09	comment	added	Konrad Rudolph		@Ben “[You claim] that PGO … is just as good as JIT optimisation; and that's not true …” – No. I claim (1) that PGO is just as good in most (not all!) cases, and (2) that the overhead a JIT inherently carries outweighs the small edge it may occasionally gain on PGO. And once again, I acknowledge that I have no empirical data to back this up and that collecting such data in a comprehensive and impartial manner is very hard, maybe impossible.
Apr 2, 2012 at 5:09	comment	added	Ben		@Patrick Doesn't inline assembler rely on compiler extensions to the language spec? I wouldn't call inline assembler C++, more of a foreign language interface. Much as many languages support including fragments of C in their source files to support interfacing with libraries written in C; by your argument I could claim that the performance of every one of those languages is equal or better than C++ in all instances. But that's useless as an attempt to look at language performance.
Apr 2, 2012 at 3:12	comment	added	Patrick87		@Ben My impression was that the question was about dynamic vs. static language performance. In any event, inline assembler is part of of the C/C++ language, right? It's not an extension... anyway, I think it's probably fine to agree to disagree. You think my approach misses the point, and I think yours relies on notions which aren't well defined. That's OK.
Apr 2, 2012 at 3:01	comment	added	Ben		@Patrick: To be honest, the performance of a JIT handwritten in assembler is not what I would describe as "C/C++ like performance", any more than I would describe the performance of a Python program that is largely a CPython extension as "Python-like performance". The question is about language implementations (i.e. the C++ compiler and the Python interpreter), not about what you can get by rewriting large amounts of your program in a lower level language that effectively bypasses that implementation.
Apr 2, 2012 at 0:49	comment	added	Patrick87		@Ben Uninteresting, maybe, but clearly true, and clearly an answer to the question. If you want to limit static languages to make the question more interesting, go for it; but you should carefully define what limitations you're assuming, and exactly what it is you're comparing. If you're talking about "usual" (=average?) statically compiled programs, are you comparing to "usual" dynamic programs, which might not make very good use of the JIT? Are dynamic programs allowed to implement JIT (or static?) compilers, if doing so would boost performance?
Apr 2, 2012 at 0:33	comment	added	Ben		@Konrad: That dynamic languages need to track runtime information anyway makes them very good candidates for dynamic runtime optimisation (and very poor candidates for static optimisation). It doesn't say anything either about whether dynamic runtime optimisation is inherently unable to make up for the overhead of tracking that runtime information. The only reason you've offered for that claim is that PGO can be done without tracking runtime information and is just as good as JIT optimisation; and that's not true, either in theory or of current PGO-ing compilers that I'm aware of.
Apr 2, 2012 at 0:27	comment	added	Ben		@Patrick87 So your argument that statically optimized C++ can always do at least as well as a JIT compiler because you can hand code a JIT compiler in C++? Technically true, but I find it uninteresting. I find it more interesting (and fair) to compare what an optimising JIT compiler can do to "normal" code and what an optimising AOT compiler can do to "normal" code. Very very few programmers will ever hand-write a JIT compiler, but very very many programmers will make use of JIT optimisation and AOT optimisation.
Mar 30, 2012 at 16:39	comment	added	Patrick87		@Ben "(or write assembler that emits specialised assembler at runtime, i.e. a JIT compiler)" I think you just refuted your own refutation, if that's what the comment was intended to be. In any event, yes, that was the idea.
Mar 30, 2012 at 10:14	comment	added	Konrad Rudolph		@Ben My argument follows from the fact that dynamic languages need to track runtime information, and efficient execution of dynamic languages necessitates a JIT with HotSpot optimisation.
Mar 30, 2012 at 3:29	comment	added	Ben		@Konrad Yes, the original question was static vs dynamic languages, but all your arguments for your "no, by design" answer stem on it costing more to track runtime information than you can possibly gain in runtime-optimisations, which is basically about JITs. I'm not convinced that your intuition is correct, but I don't know that you're wrong either.
Mar 30, 2012 at 3:19	comment	added	Ben		Likewise @Patrick87, unless you want to hand write multiple specialised versions of all your assembler (or write assembler that emits specialised assembler at runtime, i.e. a JIT compiler), it's not true that you can do anything in C/assembler ahead-of-time that a JIT language could do at runtime. And even if you are hand-specialising, you will never write every possible specialised version (and if you did it would probably blow out memory and actually perform worse), so you (or the compiler) can still be surprised.
Mar 30, 2012 at 3:15	comment	added	Ben		@Konrad: Your intuition is false. It's not about varying at runtime, it's about unpredictability at compile time. The sweet spot for JITs vs static optimisation isn't when the behaviour of the program changes at runtime "too fast" for profiling, it's when the behaviour of the program is easy to optimise in each individual run of the program, but varies wildly between runs. A static optimiser will generally have to optimise for only one set of conditions, whereas a JIT optimises each run separately, for the conditions that happen to occur in that run.
Mar 29, 2012 at 16:31	comment	added	Patrick87		@KonradRudolph Right, but the point is that you have to artificially restrict statically compiled languages in order for there to exist any meaningful comparison. Otherwise, it's rather rather like asking whether whole grains are better for you than food: on the average, this might have a meaningful answer, and for specific examples, it may go either way, but ultimately, the answer is no, since you can do at least as well with food as with whole grains, since anything you can eat that's a whole grain, you can also eat because it's food...
Mar 29, 2012 at 13:45	comment	added	Konrad Rudolph		@Patrick87 That’s a valid comment. Nevertheless, I would advance that for typical high-quality code we can make somewhat meaningful comparisons. For instance, even if you require high performance you would usually not use inline assembler in C++ – there’s simply no need to (especially since modern optimisers are pretty smart and know more about the CPU architecture than most programmers).
Mar 29, 2012 at 13:14	comment	added	Patrick87		You could, of course, define a static language which compiles to very inefficient binary, so that any real-world dynamic language would beat it; and vice versa. As I mentioned in another comment, C/C++ can trivially beat or tie any other language in terms of performance, since you can inline assembly directly into the source, and since you could replicate in C/C++ the functioning of any program written in a dynamic language (as well as the associated runtime). If you think that's cheating, let the statically compiled language be bare machine code. You could compile that by hand, if you wanted.
Mar 29, 2012 at 8:36	comment	added	Konrad Rudolph		@Ben You said “you can't even statically compile Python at all” … that’s the point: the original question wasn’t AOT vs. JIT, it was static languages vs. dynamic languages.
Mar 29, 2012 at 7:09	comment	added	Konrad Rudolph		@Ben My intuition is that there are two general classes of runtime situations: those without great variability, which are well approximated by a profile; and those which are not, but vary too fast even for the JIT to respond to properly (it first needs to collect quite a lot of profile data, after all). There certainly exists a sweet spot between the two extrema which favours JIT but my intuition is that this sweet spot is very small and doesn’t offset (both in terms of occurrence and gains) the overhead of the JIT.
Mar 29, 2012 at 0:55	comment	added	Ben		@Konrad The same sort of principle will apply to any program whose operations are controlled by outside input (if you like, anything "interpreter-like" for a very wide definition of "interpreter", that includes web-servers for example). JIT compilation will detect "hotspots" that actually occur this time the program is run. PGO will detect "hotspots" that occurred in the profile. How do you profile an application whose behaviour varies in an unbounded manner based on runtime information?
Mar 29, 2012 at 0:49	comment	added	Ben		@Konrad Take PyPy's meta-tracing JIT. It works at the level of the interpreter's code, not at the level of the Python program being run by the interpreter. It effectively optimises the operations of the interpreter, specialised to the user-input Python program it happens to be running. No amount of profile-guided optimisation of the PyPy interpreter could do anything like that. And you can't even statically compile Python at all, let alone PGO it.
Mar 28, 2012 at 22:26	comment	added	Konrad Rudolph		@Ben I don’t deny that it’s complicated. This is merely an intuition. Tracking all that information at runtime comes at a cost, after all. I’m not convinced by your point about IO. If this is predictable (= typical use-case), then PGO can predict it. If it’s spurious, I’m not convinced that the JIT could optimise it either. Maybe once in a while, out of sheer luck. But reliably? …
Mar 28, 2012 at 22:21	comment	added	Ben		Plus, there really are optimisations that can never be applied by an ahead-of-time compiler. Any "hotspot" that depends on information that is only gathered at runtime (IO basically) can in principle be JIT-optimised for the specific case at hand, but even a profile-based ahead-of-time optimiser is never going to be able to do that. I don't know that your supposition is wrong, but I don't find your arguments conclusive; it's a much more complicated topic than that.
Mar 28, 2012 at 22:12	comment	added	Ben		However, it's a bit of an apples-oranges situation. You wouldn't make a JIT compiler for C, for the reasons you state. Much information is available ahead of time, so there's less left to gain at runtime, and it would be hard to make the additional overhead less than the gain. For dynamic languages though, most of the information is already being tracked just to implement the semantics of the language, so the overhead is less and the potential payoff bigger.
Mar 28, 2012 at 22:07	comment	added	Ben		You offer an interesting point about profile-guided optimisations. I'm not aware that any existing profile-based optimising compilers emit multiple specialised versions of the code they compile (i.e. after seeing a profile where 30% of the time one code path is taken through a function and 25% of the time another path is taken, plus the fully general version to handle anything else), which JIT compilers do automatically. But in theory, yes, with sufficiently detailed profiles there's no reason C++ compilers couldn't do this.
Mar 28, 2012 at 14:22	comment	added	Konrad Rudolph		@Raphael The key word is: maybe. It doesn’t show it either way. That is all I wanted to say. I have given reasons (but no proof) for my assumption in the preceding paragraphs.
Mar 28, 2012 at 14:21	comment	added	Raphael		No, it does not show that. Maybe JIT is better than statically compiled code. You do not give reasons for your assumption (static compiler produces faster code than JIT), you take it as granted. Without this assumption, you can not make the statement about `javac`.
Mar 28, 2012 at 14:16	comment	added	Konrad Rudolph		@Raphael That’s why I put it in quotes. As the rest of the comments explains, I don’t actually consider it a shortcoming. But in principle my argument is (or at least I’m not convinced of the opposite, despite regular insistence) that a static compiler (with PGO) can produce more efficient programs than a JIT, if we include the cost of keeping track of runtime program flow information. And the fact that `javac` never performed as well as the JIT doesn’t invalidate my point, it just shows that `javac` never optimised that well.
Mar 28, 2012 at 13:27	comment	added	Raphael		I can not say how good `javac` ever was. However, saying "That’s a “shortcoming” of the compiler then." is not an argument; the statement already assumes you are right.
Mar 28, 2012 at 13:10	comment	added	Konrad Rudolph		@Raphael That’s a “shortcoming” of the compiler then. In particular, did `javac` ever profile-guided optimisation? Not as far as I’m aware of. In general it doesn’t make sense to make the compiler of a JITted language well at optimising since the JIT can handle it (and at the very least, this way more languages profit from the effort). So (understandably) there never was put much effort into the `javac` optimiser, as far as I know (for the .NET languages this is definitely true).
Mar 28, 2012 at 12:38	comment	added	Raphael		Me too. `javac` used to do that stuff but does not anymore (as JIT is hampered). The observation is that unoptimised (byte-)code with JIT is faster than optimsed (byte-)code.
Mar 28, 2012 at 12:26	history	edited	Konrad Rudolph	CC BY-SA 3.0	Clarification of the last paragraph.
Mar 28, 2012 at 12:22	comment	added	Konrad Rudolph		@Raphael I meant: compared to static analysis and optimisation.
Mar 28, 2012 at 11:32	comment	added	Raphael		Your last sentence is far too strong. Bookkeeping clearly pays off as evidenced when comparing Java w/o JIT against Java w/ JIT.
Mar 28, 2012 at 10:16	history	answered	Konrad Rudolph	CC BY-SA 3.0

toggle format