84
$\begingroup$

I have noticed that some applications or algorithms that are built on a programming language, say C++/Rust run faster or snappier than those built on say, Java/Node.js, running on the same machine. I have a few question regarding this:

  1. Why does this happen?
  2. What governs the "speed" of a programming language?
  3. Has this anything to do with memory management?

I'd really appreciate if someone broke this down for me.

$\endgroup$
97
$\begingroup$

In programming language design and implementation, there is a large number of choices that can affect performance. I'll only mention a few.

Every language ultimately has to be run by executing machine code. A "compiled" language such as C++ is parsed, decoded, and translated to machine code only once, at compile-time. An "interpreted" language, if implemented in a direct way, is decoded at runtime, at every step, every time. That is, every time we run a statement, the intepreter has to check whether that is an if-then-else, or an assignment, etc. and act accordingly. This means that if we loop 100 times, we decode the same code 100 times, wasting time. Fortunately, interpreters often optimize this through e.g. a just-in-time compiling system. (More correctly, there's no such a thing as a "compiled" or "interpreted" language -- it is a property of the implementation, not of the language. Still, each language often has one widespread implementation, only.)

Different compilers/interpreters perform different optimizations.

If the language has automatic memory management, its implementation has to perform garbage collection. This has a runtime cost, but relieves the programmer from an error-prone task.

A language might be closer to the machine, allowing the expert programmer to micro-optimize everything and squeeze more performance out of the CPU. However, it is arguable if this is actually beneficial in practice, since most programmers do not really micro-optimize, and often a good higher level language can be optimized by the compiler better than what the average programmer would do. (However, sometimes being farther from the machine might have its benefits too! For instance, Haskell is extremely high level, but thanks to its design choices is able to feature very lightweight green threads.)

Static type checking can also help in optimization. In a dynamically typed, interpreted language, every time one computes x - y, the interpreter often has to check whether both x,y are numbers and (e.g.) raise an exception otherwise. This check can be skipped if types were already checked at compile time.

Some languages always report runtime errors in a sane way. If you write a[100] in Java where a has only 20 elements, you get a runtime exception. In C, that would cause undefined behavior, meaning that the program might crash, overwrite some random data in memory, or even perform absolutely anything else (the ISO C standard poses no limits whatsoever). This requires a runtime check, but provides a much nicer semantics to the programmer.

However, keep in mind that, when evaluating a language, performance is not everything. Don't be obsessed about it. It is a common trap to try to micro-optimize everything, and yet fail to spot that an inefficient algorithm/data structure is being used. Knuth once said "premature optimization is the root of all evil".

Don't underestimate how hard it is to write a program right. Often, it can be better to choose a "slower" language which has a more human-friendly semantics. Further, if there are some specific performance critical parts, those can always be implemented in another language. Just as a reference, in the 2016 ICFP programming contest, these were the languages used by the winners:

1   700327  Unagi                       Java,C++,C#,PHP,Haskell
2   268752  天羽々斬                     C++, Ruby, Python, Haskell, Java, JavaScript
3   243456  Cult of the Bound Variable  C++, Standard ML, Python

None of them used a single language.

$\endgroup$
  • 81
    $\begingroup$ One version of the whole quote from Knuth: "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." $\endgroup$ – Derek Elkins Mar 24 '17 at 9:36
  • 7
    $\begingroup$ Also some languages guarantee seemingly innocent things that can affect the ability of the compiler to optimize, see strict aliasing in C++ vs. FORTRAN $\endgroup$ – PlasmaHH Mar 24 '17 at 11:03
  • 10
    $\begingroup$ Joined so I could upvote the comment by @DerekElkins. Too often the context of that quote is missing, sometimes even leading to the conclusion that it advocates that all optimization is evil. $\endgroup$ – pipe Mar 24 '17 at 12:36
  • 9
    $\begingroup$ @DerekElkins Ironically, you missed perhaps the most important part: the two sentences immediately following. "A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail." PDF p268 $\endgroup$ – 8bittree Mar 24 '17 at 14:58
  • 9
    $\begingroup$ @DerekElkins To be clear, I don't want to put words in your mouth say you were claiming this, but too often I come across people adding just a little bit to the base quote, and thinking that Knuth was advocating for premature optimization 3% of the time, when the full article makes it clear he always opposes premature optimization, almost always opposes small optimizations, but advocates for small optimizations in sections measured as critical. $\endgroup$ – 8bittree Mar 24 '17 at 15:08
19
$\begingroup$

What governs the "speed" of a programming language?

There is no such thing as the "speed" of a programming language. There is only the speed of a particular program written by a particular progammer executed by a particular version of a particular implementation of a particular execution engine running within a particular environment.

There can be huge performance differences in running the same code written in the same language on the same machine using different implementations. Or even using different versions of the same implementation. For example, running the exact same ECMAScript benchmark on the exact same machine using a version of SpiderMonkey from 10 years ago vs a version from this year will probably yield a performance increase anywhere between 2×–5×, maybe even 10×. Does that then mean that ECMAScript is 2× faster than ECMAScript, because running the same program on the same machine is 2× faster with the newer implementation? That doesn't make sense.

Has this anything to do with memory management?

Not really.

Why does this happen?

Resources. Money. Microsoft probably employs more people making coffee for their compiler programmers than the entire PHP, Ruby, and Python community combined has people working on their VMs.

For more or less any feature of a programming language that impacts performance in some way, there is also a solution. For example, C (I'm using C here as a stand-in for a class of similar languages, some of which even existed before C) is not memory-safe, so that multiple C programs running at the same time can trample on each other's memory. So, we invent virtual memory, and make all C programs go through a layer of indirection so that they can pretend they are the only ones running on the machine. However, that is slow, and so, we invent the MMU, and implement virtual memory in hardware to speed it up.

But! Memory-safe languages don't need all that! Having virtual memory doesn't help them one bit. Actually, it's worse: not only does virtual memory not help memory-safe languages, virtual memory, even when implemented in hardware, still impacts performance. It can be especially harmful to the performance of garbage collectors (which is what a significant number of implementations of memory-safe languages use).

Another example: modern mainstream general purpose CPUs employ sophisticated tricks to reduce the frequency of cache misses. A lot of those tricks amount to trying to predict what code is going to be executed and what memory is going to be needed in the future. However, for languages with a high degree of runtime polymorphism (e.g. OO languages) it is really, really hard to predict those access patterns.

But, there is another way: the total cost of cache misses is the number of cache misses multiplied by the cost of an individual cache miss. Mainstream CPUs try to reduce the number of misses, but what if you could reduce the cost of an individual miss?

The Azul Vega-3 CPU was specifically designed for running virtualized JVMs, and it had a very powerful MMU with some specialized instructions for helping garbage collection and escape detection (the dynamic equivalent to static escape analysis) and powerful memory controllers, and the entire system could still make progress with over 20000 outstanding cache misses in flight. Unfortunately, like most language-specific CPUs, its design was simply out-spent and out-brute-forced by the "giants" Intel, AMD, IBM, and the likes.

The CPU architecture is just one example that has an impact on how easy or how hard it is to have a high-performance implementation of a language. A language like C, C++, D, Rust that is a good fit for the modern mainstream CPU programming model will be easier to make fast than a language that has to "fight" and circumvent the CPU, like Java, ECMAScript, Python, Ruby, PHP.

Really, it's all a question of money. If you spend equal amounts of money to develop a high-performance algorithm in ECMAScript, a high-performance implementation of ECMAScript, a high-performance operating system designed for ECMAScript, a high-performance CPU designed for ECMAScript as has been spent over the last decades to make C-like languages go fast, then you will likely see equal performance. It's just that, at this time, much more money has been spent making C-like languages fast than making ECMAScript-like languages fast, and the assumptions of C-like languages are baked into the entire stack from MMUs and CPUs to operating systems and virtual memory systems up to libraries and frameworks.

Personally, I am most familiar with Ruby (which is generally considered to be a "slow language"), so I will give two examples: the Hash class (one of the central data structures in Ruby, a key-value dictionary) in the Rubinius Ruby implementation is written in 100% pure Ruby, and it has about the same performance as the Hash class in YARV (the most widely-used implementation), which is written in C. And there is an image manipulation library written as a C extension for YARV, that also has a (slow) pure Ruby "fallback version" for implementations that don't support C which uses a ton of highly-dynamic and reflective Ruby tricks; an experimental branch of JRuby, utilizing the Truffle AST interpreter framework and Graal JIT compilation framework by Oracle Labs, can execute that pure Ruby "fallback version" as fast as the YARV can execute the original highly-optimized C version. This is simply (well, anything but) achieved by some really clever people doing really clever stuff with dynamic runtime optimizations, JIT compilation, and partial evaluation.

$\endgroup$
  • 4
    $\begingroup$ While technically languages are not inherently fast, some languages have a greater focus on allowing the programmer to make fast code. C is primarily optimised to CPUs rather than the other way round. For example, C chose fixed size ints for performance reasons, despite the fact that unbounded integers such as those used by Python are much more mathematically natural. Implementing unbounded integers in hardware wouldn't be as fast as fixed sized integers. Languages that try to hide implementation details need complex optimisations to come close to naive C implementations. $\endgroup$ – gmatht Mar 26 '17 at 1:50
  • 1
    $\begingroup$ C has a history - it was created to make a system written in assembly language portable to other systems. So the original purpose was to provide a "portable assembler" for Unix, and was very well designed. It did so well from 1980-1995-ish that it had critical mass when Windows 95 came around. $\endgroup$ – Thorbjørn Ravn Andersen Mar 26 '17 at 21:59
  • 1
    $\begingroup$ I would disagree with the comment about resources. It's not the number of people on the team that matters, it's the skill level of the best people on the team. $\endgroup$ – Michael Kay Mar 26 '17 at 22:24
  • $\begingroup$ "Microsoft probably employs more people making coffee for their compiler programmers than the entire PHP, Ruby, and Python community combined has people working on their VMs" I suppose that depends on how far you're willing to stretch the term "compiler programmer" and how much you're including with that (Microsoft develops a lot of compilers). For example, just the VS C++ compiler team is relatively small AFAIK. $\endgroup$ – Cubic May 4 '18 at 13:40
7
$\begingroup$

Theoretically, if you write code that does exactly the same in two languages and compile both using a "perfect" compiler, the performance of both should be the same.

In practice, there are several reasons why the performance is going to be different:

  1. Some languages are harder to optimize.

    This includes especially features that make code more dynamic (dynamic typing, virtual methods, function pointers), but also for example garbage collection.

    There are various ways to make code using such features fast, but it usually still ends up at least somewhat slower than without using them.

  2. Some language implementations have to do some compiling at runtime.

    This applies especially to languages with virtual machines (like Java) and languages that execute the source code, with no intermediate step for binary (like JavaScript).

    Such language implementations have to do more work at runtime, which affects performance, especially startup time/performance soon after startup.

  3. Language implementations intentionally spend less time on optimizations than they could.

    All compilers care about performance of the compiler itself, not just of the generated code. This is especially important for runtime compilers (JIT compilers), where taking too long to compile slows down the application execution. But it applies to ahead-of-time compilers, like those for C++ too. For example register allocation is an NP-complete problem, so it's not realistic to solve it perfectly and instead heuristics that execute in reasonable time are used.

  4. Differences in idioms in different languages.

    Code written in the style common for a certain language (idiomatic code) using common libraries can result in difference in performance, because such idiomatic code behaves subtly differently in each language.

    As an example, consider vector[i] in C++, list[i] in C# and list.get(i) in Java. The C++ code likely does no range checking and performs no virtual calls, the C# code performs range checking, but no virtual calls and the Java code performs range checking and it's a virtual call. All three languages support virtual and non-virtual methods and both C++ and C# can include range checking or avoid it when accessing the underlying array. But the standard libraries of these languages decided to write these equivalent functions differently, and, as a consequence, their performance will be different.

  5. Some compilers might be missing some optimizations.

    Compiler writers have finite resources, so they can't implement every possible optimization, even ignoring the other issues. And the resources they have might be focused on one area of optimization more than others. As a consequence, code written in different languages can have different performance because of differences in their compilers, even if there is no fundamental reason why one language should be faster or even easier to optimize than the other.

$\endgroup$
  • $\begingroup$ Some language don't have a compiler. For some languages, there can't be a compiler (reference). $\endgroup$ – Raphael Mar 24 '17 at 18:16
  • 3
    $\begingroup$ For some languages, there can't be a static compiler. Dynamic compilation isn't affected by the undecidability of static properties. $\endgroup$ – Jörg W Mittag Mar 24 '17 at 21:06
  • $\begingroup$ If languages are different enough, they don't have "code which does exactly the same". They might have code which produces the same output or user-observable behavior, which is not the same thing. So I disagree with your premise. $\endgroup$ – einpoklum - reinstate Monica Mar 24 '17 at 21:47
  • $\begingroup$ @einpoklum I don't see the distinction. With some exceptions (like threading synchronization), which I think are not interesting here, language specifications usually allow any optimizations that preserve observable behavior. What kind of behavior do you have in mind that is not user-observable, but cannot be optimized? $\endgroup$ – svick Mar 24 '17 at 22:38
  • $\begingroup$ Theoretically, any interpreted language can be “compiled” by generating an EXE file consisting of an interpreter + the program's source code. $\endgroup$ – dan04 Mar 24 '17 at 23:21
1
$\begingroup$

This is a very general question, but in your case the answer could be simple. C++ compiles to machine code, where Java compiles to Java byte code, which is then run of a Java virtual machine (though there is also just-in-time compiling, which further compiles the Java byte code to native machine code). Another difference could be garbage collection, which is a service that only Java provides.

$\endgroup$
  • 2
    $\begingroup$ This is too simplistic an answer. There are settings in which Java is faster. $\endgroup$ – Raphael Mar 24 '17 at 18:15
  • 4
    $\begingroup$ There are also implementations of Java which compile to machine code and interpreted implementations of C++. And what is "machine code" anyway, if I have a picoJava CPU which executes JVM bytecode natively, and I have the JPC x86 interpreter running on the JVM, then what makes x86 object code "machine code" and JVM bytecode not? $\endgroup$ – Jörg W Mittag Mar 24 '17 at 21:08
  • 2
    $\begingroup$ Nitpick: Not only Java provides garbage collection... and even if you only consider C++ and Java, some C++ frameworks/libraries programs have garbage collection facilities in place. $\endgroup$ – einpoklum - reinstate Monica Mar 24 '17 at 21:48
  • $\begingroup$ Compiling to native machine code generally improves performance. However, in some limited cases a high-quality interpreter can beat out a naive just-in-time compiler. $\endgroup$ – DepressedDaniel Mar 25 '17 at 6:21
  • $\begingroup$ @JörgWMittag The trick is to compile to native machine code - the machine code that the host processor understands - and then directly jump to the called code so it can be executed without interpretation. This will be x86 on an x86 chip and JVM bytecodes on a picoJava CPU. $\endgroup$ – DepressedDaniel Mar 25 '17 at 6:23
0
$\begingroup$

Simple answer is, the closer a language is to the hardware the faster it would be.

The more a language depends on libraries and other functions to do it's tasks the longer it takes for program execution in that language.

$\endgroup$
0
$\begingroup$

It bears repeating that the algoritm used is most important.

An example might be you writing your oen bubble sort on a million records in the worlds fastest language. Compared it to using a really slow language with a built in really good sort. Your fast language will be mighty slow in comparison.

$\endgroup$
-5
$\begingroup$

Your question is too general, so I'm afraid I can't give an exact answer that you need.

For my best explanation, let's look at C++ and .Net platform.

C++, very close to machine code and one of the favor programming in the older time (like more than 10+ years ago?) because of the performance. There isn't much resources needed to develop and execute C++ program even with the IDE, they are consider very light weight and very fast. You can also write C++ code in console and develop a game from there. In terms of memory and resources, I forgotten roughly how much capacity it takes in a computer but the capacity is something that you can't compare with current generation of programming language.

Now let's look at .Net. The prerequisite to do .Net development is one giant IDE that consists of not just one type of programming languages. Even if you just want to developer in let's say C#, the IDE itself will includes many programming platforms by default such as J#, VB, mobile and etc by default. Unless you do custom install and only install exactly what you want, provided that you are experience enough to play with the IDE installation.

Other than installing the IDE software itself, .Net also comes with huge capacity of libraries and frameworks for the purpose of ease of access to any kind of platform that developers need AND don't need.

Developing in .Net can be a fun experience because many common function and components are available by default. You can drag-and-drop and use many validation method, file reading, database access and much more in the IDE.

As a result, it's a trade off between computer resources and speed of development. Those libraries and framework takes up memory and resources. When you execute a program in .Net IDE, it may consume tons of time trying to compile the libraries, components and all of your files. And when you do debugging, your computer require lots of resources to manage the debugging process in your IDE.

Usually, in order to do .Net development, you need a good computer with the some Ram and processor. Otherwise, you might as well don't programming at all. In this aspect, C++ platform is much better than .Net. Although you still need a good computer, but capacity won't be too much of concern compare to .Net.

Hope my answer could help with your question. Let me know in case you want to know more.

EDIT:

Just a clarification to the my main point, I mainly answer to the question of "What's govern the speed of programming".

In IDE standpoint, using C++ or .Net language in the relative IDE does not affect the speed of writing the code, but it will affect the speed of development because Visual Studio compiler take longer time to execute the program, but C++ IDE is much lighter and consume less computer resources. So in the long run, you can do programming faster with C++ type of IDE compare with .Net IDE which heavily depending on libraries and framework. If you take in the amount of time of waiting the IDE to start up, compile, execute the program and etc, it will affect the speed of programming. Unless "the speed of programming" is actually only focus on "speed of writing code".

The amount of libraries and framework in Visual Studio also consume the computer capacity. I'm not sure if this answer the question of "memory management", but I just want to point out that Visual Studio IDE can take up lots of resources when running it, and thus slow down the overall "speed of programming". Of course, this has nothing to do with "speed of writing code".

As you may have guessed, I don't know too much of C++ as I just use it as example, my main point is about Visual Studio type of heavy IDE that consume computer resources.

If I got the idea and did not answer the thread starter question at all, apologize for the long post. But I would advice the thread starter to make the question clear and ask exactly what he/she need to know about "faster" and "slower"

$\endgroup$
  • 6
    $\begingroup$ Just for the record, .NET development only requires the .NET SDK (and, for execution, the .NET runtime itself). You can use a plain text editor and invoke the compiler from the command line. The IDE (I assume you are referring to Visual Studio) is NOT required, although it sure helps with any sizable project (Intellisense, debugger, etc etc). There are also lighter IDEs such as SharpDevelop with less bells and whistles. $\endgroup$ – MetalMikester Mar 24 '17 at 12:33
  • 16
    $\begingroup$ You can certainly develop in .Net without an IDE. But I don't see how is the IDE relevant to the speed of code written in a language. $\endgroup$ – svick Mar 24 '17 at 12:37
  • 8
    $\begingroup$ @MetalMikester: And of course, Visual Studio is the go-to IDE for C++ development on Windows. $\endgroup$ – Jörg W Mittag Mar 24 '17 at 12:42
  • 3
    $\begingroup$ And compiling C++ to .Net code is just a single compiler switch; the supposed chasm is bridges by one mouse click in visual studio. $\endgroup$ – MSalters Mar 24 '17 at 16:00
  • 1
    $\begingroup$ I'm not at all sure I would call modern C++ "very close to machine code". Original C, yes; it was conceived as a portable assembler and has remained very close to that (though the standard library has grown, and various compilers offer add-ons to the language proper that takes it farther from its origins). Original C++ (C With Classes), maybe; rewriting a variant of C that includes classes to pure C isn't that difficult, at which point the previous discussion applies. But modern C++, with templates, polymorphism, and everything else under the Sun? That's hardly "very close to machine code". $\endgroup$ – a CVn Mar 27 '17 at 8:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.