Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. It only takes a minute to sign up.
Sign up to join this community
I assume that computers make many mistakes (like errors, bugs, glitches, etc.), which can be observed from the amount of questions asked everyday on different communities (like Stack Overflow) showing people trying to fix such issues.
If computers really make many errors (as I assumed earlier) then critical tasks (like signing in or receiving a receipt) must be designed to be almost error-free, unlike most of the tasks of most software and video games.
If computers really make many errors (as I assumed earlier)...
Your assumption is wrong. Firstly, computers do not (except in extreme cases) "make many errors", humans do. Computers simply do what they are told to, very quickly and very well. Extraordinarily well, all things considered.
Secondly, what you're perceiving as a high rate of errors is in reality a high perception of errors. A banking system that handles millions of transactions per day may have a hidden bug (a human error in coding) that in a very specific set of circumstances does something incorrect. Those circumstances may only occur after years of correct operations. One failure in several billion operations is not a high error rate, but you hear about the failure and assume that the banking system must be error prone.
Back in the days when humans handled all the transactions - way, way back when that was even possible - the average error rate for transactions was many orders of magnitude higher. You can see that today in accounting circles, where end-of-month processing routinely involves tracking down human data entry errors to figure out why the (electronically recorded and processed) books don't balance.
then critical tasks (like signing in or receiving a receipt) must be designed to be almost error-free
While signing into a system is a relatively simple thing to implement, the big-ticket flaws you've heard about generally come from outside of your own implementation. OpenSSL's Heartbleed vulnerability potentially allowed an attacker to snoop on your secure communications, Spectre/Meltdown potentially allows malware to snoop on your program's memory, and there are all sorts of interesting attacks on encryption that make life difficult. And any time you hear about one of these things it's sensational, front-page news on the tech sites and a flurry of "how do I protect against this" posts on programming sites.
And yet billions of logins happen daily, more billions of transactions get processed, making the true failure rate almost absurdly low.
unlike most of the tasks of software and video games.
Software development for critical systems is an entirely different environment than game development. For one thing, only one of the two is about making money as quickly and cheaply as possible. Games developers don't actually care if a few bugs go through to production, because who cares if your car clips through walls in an out-of-the-way part of the map, or in some rare case manages to slide through the ground and you end up falling out of the map... just reload a save or something, and we'll patch that later. After all, modern gamers are used to being used as alpha testers on games, just as long as they get to have some fun who cares? Especially if they're paying for the privilege. And it's not like you have the time or the budget to rigorously test every possible code path.
Critical systems development can't be that sloppy. Every element is tested thoroughly, ever combination of elements is further tested. A lot of very bright people are paid a lot of money to find new ways to torture your code to make it break down and produce errors, and when they succeed it's back to the dev team to find out whay it broke and fix that issue. The customer doesn't get to see the system in action until it has been tested in every conceivable way. Because when it fails really bad things happen, and save points are really far apart in the real world.
Why do we rely on computers in critical fields?
The simple reality is that there is no better option available to us. The things we use computers for simply can't be handled by humans alone in the volume, speed and accuracy required.
Are there alternatives? Maybe. Analog computing is great for some things, quantum computing offers some interesting possibilities. But for right now general-purpose computing is the best option on so many levels, and it's what we've spent most of the last century focusing on.
Computer hardware almost always does exactly what software tells it to do.
It's useful to distinguish software bugs from unreliable hardware.
Cosmic rays can randomly flip a bit in memory, though; that's why servers often use ECC (Error Correction Code) memory to correct single-bit errors and detect most multi-bit errors. (And internally, CPUs usually use ECC for their caches.)
Computers that need even more reliability and availability1 than that, like flight computers in aircraft or space craft, often have 3 separate computers processing the same inputs. (Triple Modular Redundancy) If all 3 produce the same output, great, it's almost certainly correct. (Especially if each of the 3 computers is running software written by different teams.) If only 2 out of the 3 outputs match, the odd one out is assumed wrong, so it gets reset and the system uses the outputs of the remaining two until the faulty one is rebooted and agreeing with them. If all 3 systems give different outputs, you have a big problem. If 2 systems give the same wrong answer, that's even worse.
Safety-critical systems like those are programmed in software much more carefully than video games or even mainstream OSes. Practices like avoiding dynamic memory allocation (malloc) remove whole classes of bugs and possible corner cases.
Footnote 1: detecting an error and rebooting is not sufficient when the system is part of the flight controls of a jet plane that could crash if the controls stopped responding for half a second.
Related:
Most errors in computer systems are caused by humans directly through design and programming errors.
Almost all questions on Stack Overflow concern programming problems and most errors that make it to end users are software bugs (human oversight).
Logic errors in hardware design (human oversight) have also caused trouble but theses errors are less common. Such issues can normally only be worked around at the OS or compiler level as hardware is almost impossible to fix once shipped.
Finally, as computers are electronic devices, they are susceptible to environmental influences (such as heat or radiation) and undesired complex physical interactions. Electrical interference can, for example, cause bit flips in DRAM but this is extremely rare as far as we are aware. It turns out such bit flips can even be caused intentionally from malicious software, which is known as row hammering, and it is believed by some that certain software might in rare instances cause bit flips by accident.
While the first two kinds of errors can be avoided through careful testing, simulation and formal modeling and verification, it is almost impossible to build an accurate physical model of an entire computer and all its smallest building blocks. As such, we will almost never be able to build a 100% reliable computer that works in uncontrolled environments. Nonetheless, finding better ways to build correct software through modeling and verification will already make computers much more reliable.
Because there's no practical alternative. Modern life is too complex for ordinary humans to keep track of all the details. And critical systems often require reaction times far quicker than humans or mechanical systems are capable of.
Consider a spacecraft. It likely has thousands of components that have to be monitored constantly. Mission Control would have to have hundreds of people on each shift watching all these readings, and they still wouldn't be as reliable as the computers.
Or in a hospital ICU, what would you replace all the computerized monitors with? You can't have a nurse watching each patient. We need to automate this, and have the computers raise alarms when a reading goes to an abnormal value.
The old saying "Don't let the perfect be the enemy of the good" applies. Despite the potential problems, computers are good enough to do what we need of them. There are millions of cars on the road with computerized control systems in the engines -- when have you heard of any of them failing?
Computers can have bugs, but safety-critical systems go through extensive testing. When computer software fails, the bugs can be fixed or the equipment can be replaced; it's harder to fix human error (you can do additional training, but people have inherent limits on speed and concentration).
There are other ways to mitigave computer problems, such as redundancy. The Space Shuttle had 5 identical computers operating the avionics system. During critical mission phases, 4 of them would would operate together, each calculating a result, and they would "vote" on the output. See Redundancy Management Technique for Space Shuttle Computers. Some airliners use redundant computers with completely different hardware and independently-written software, to avoid the possiblility that the same bug will be in both; see How dissimilar are redundant flight control computers?
Of course, less critical systems do fail more often. A few days ago the display of my car's navigation and entertainment system froze after displaying the startup message about using the system safely. Restarting the car rebooted the system and it was fine then. This system isn't necessary for driving, so it wouldn't have been a problem if it happened during high speed driving when I couldn't stop and start the car.
Basically hardware is reliable because it has to. It's not possible to fix a chip, and it's either impossible or not economical to fix a circuit board after it has left the factory. If a company sells buggy hardware, they can expect customers to ask for a refund.
In addition, hardware and especially chips have huge upfront costs. If the design for a cpu makes it to manufacture with bugs, someone is going to lose a bucketload of money and be very displeased about it.
Therefore, hardware and chip companies had to come up with ways to ensure the product actually works. And when there's a bug and the product is sold anyway, you can expect full documentation, erratas and workarounds.
On the other hand, no-one cares about software. It's kinda expected that it's going to be full of bugs, and it's easy to fix them with an update, which costs very little. The huge cost of throwing away bad chips and redoing a fabrication run does not exist for software.
Unless the software runs something like a rocket, or a plane, or a fabrication plant, where bugs cost a bucketload of money, there is really no incentive to make software that actually works 100% free of bugs. In fact, "release early with bugs" is often the more profitable option...
On top of the incentives, it's easier to test hardware because it is simpler. It has less state. The number of flops in a design is known. Hardware rarely manages complex data structures, buffers are usually of fixed size, synchronization is handled by well characterized primitives like FIFOs, there are much less object classes, less genericity, etc. It is designed as small pieces that can be unit tested (I heard that also exists for software, but who does that?...). And the tools pretty much guarantee that what is synthetized will do what the HDL says it should do.
The way to make sure a class of bugs doesn't happen is to find a provably correct solution and automate it. For example, if a language supports automatic memory allocation, then there can't be memory allocation bugs. It's the same for many classes of bugs. Can't have buffer overruns in a language that actually knows the size of a buffer. That kind of stuff. But using a new language is complicated, due to mountains of legacy code. This is different from hardware, where each big new generation of CPU tends to throw away large parts of the previous designs.
Now for example, SQL injections. There's one way to do it right, and that's to use a library or language that does it properly, where you pass a SQL query with placeholders and variables separately, like python DBAPI does. This is such an obvious fact, that it should not even need mentioning, yet many languages have (or are still using) the old bug-prone way of putting raw strings into queries.
A tool like PHP, which consistently produces bugs, would not live for long in the world of hardware. It would not be profitable. Yet PHP still thrives, on the software side.
Extra:
There was a time where software had a similar "cost per bug" as hardware, that was before the internet. Back in the day, bugs could not be fixed by an easy auto-update download. Sometimes, you had to mail floppies... and in the worst case, like arcade machines, game consoles, software that is burned into mask ROM, it was impossible to modify. These had the same cost incentives as hardware to get it right. These must have been thoroughly tested. Software was also much simpler back in the day, of course.
First of all nothing is unfailable. And if it be only, that its hit by a meteorite and gets completely destroyed.
Under the assumption, that there are no external factors, such as power failures, excessive heat, flooding, cosmic rays, etc. a computer will do exactly what you tell it to do.
As mentioned in other answers, this is the main crux of computers. If you do not give it clear enough constructions, there's room for error or malfunction, but this solely based on human error.
Testing exists for this purpose, but testing alone is not enough. You need to make sure, your tests are worth your money. There is a famous joke among programmers:
A programmer programs a bartender. He tests it by ordering a beer, ordering 20 beers, ordering 0 beers, ordering milk and ordering pretzels. Convinced that his software works, he opens the bar. The first guest comes in and asks for the restroom. The bar burns down in flames.
The punchline (forgive me for explaining it) is, that the programmer did not anticipate such input. As such, especially critical systems need to be guarded against all possible forms of failures in order to avoid a crush or bug. This way, a much higher reliability can be reached than letting a human do the operations.
Humans get tired, distracted, lazy. There have been too many incidents to count, where people did not bother to follow safety instructions completly (even in super critical sectors as the flight industry) which ultimately lead to disaster. Computers will follow your orders to the letter. It's on humans to account for all possible options.
While this may sound impossible (after all, aren't there infinite possibilities) let me conclude this with a small example.
Suppose you write a program that wants to read in a number greater than zero. The user could input 0, a negative number, a letter, a picture, whatever. So if your code looks like this:
user_input = get_user_input();
result = 1 / square_root(user_input);
the last line could become problematic and indeed the program will most likely crash if incorrect input was entered. So to guard it, you'd write
user_input = get_user_input();
result = default_value;
if( is_valid_number(user_input) ){
result = 1/square_root(user_value);
}
else{
print("Error: Please enter a number > 0");
}
As such, it is very much possible to build very reliable software, even with human error factors.
Computers aren't the source of the failure. Programming isn't the source of the failure.
Algorithms thought of by humans are the source of the failures.
If we get a human to do steps to "login" or "provide receipts", they will make as many, or more, errors than the program that replaces them.
In fact, when you take a set of clear steps and automate them, you get a massive reduction in process errors. This can be so large that you make a different kind of error -- you try to apply the process in cases when it isn't very reliable!
When someone signs into a hotel, for example, the front desk does a bunch of steps involving verifying their identity, checking the payment method, ensuring the room is clean, etc. Prior to computers this was all done without them, using paper records and steps the people involved where supposed to follow.
It was slow, extremely error prone, and cost a bunch of money.
Replacing the steps that humans are bad at with computers doing them made it faster, less error prone, and much cheaper. But when they did this, they also tended to implicitly fold in tasks the computers where not good at into the same workflow, like dealing with exceptions and the like.
So you can get powerless front desk staff who are unable to make the computer make an exception when it is in the business interests of the owner of the hotel and computer to let them do it. And when the computer does make a mistake, there isn't the skills or ability to route around it and fix it.
Computers have replaced entire office buildings full of paper-shufflers. Forms getting mailed in, processed following certain algorithms, totals added up and written on other forms, summaries generated and put on other forms. Typists typing up documents, sometimes in triplicate or more. Secretaries taking memos and categorizing them by importance and presenting them to their executive. All steps we now do with computers for far less cost and far faster.
The computer doesn't always do it better, but it does it with far, far less errors.
The result of this lower error rate is that we pile more systems on top of systems. If you had something that failed 99% of the time, you can't build much on top of it; but you drop that failure rate to 0.001%, you can now treat it as reliable. So you build a system on top of it and 1000 other systems; and now your failure rate is upwards of 1%. You keep doing this, generating value, until the failure rate is high.
The only super complex systems we have are either the result of extensive engineering, like cars or rockets or integrated circuits or manufacturing line or bridges, or are the result of computer programming. Of these, computer programming is by far the cheapest one to prototype and modify and extend.
With this cheap extension and prototyping, you get these insanely complex systems built on a budget that solve problems. But the cost to make them reliable is very high, much higher than the cost to build them and see if they work. So computer programs are almost always written the be cheap and fast.
There is a whole pile of computer programs that are not written this way. Software to control air planes and rockets, for example. And software for medical systems. But even there, the temptation remains; you could either make something cost 100,000x as much and be utterly reliable, or be 100,000x cheaper and fail "very rarely". There is a lot of economic pressure to save a 100,000x cost increase and justify the cheaper way of writing software.
Modern software engineering is the economic management of insanely complex systems. When and where should you spend efforts on making something reliable, and when should you accept the insane cost savings of not ensuring reliability? Every spot you can accept bugs in is a spot that is much cheaper, but also a possible reason your entire project won't work.
Computers aren't the source of the failure. this isn't true, hardware do sometimes fail (bad sector on harddisk, computer fan jammed, a piece of very rusted metal in motherboard, ect).
$\endgroup$
As others have data to show correctness in computers, I wanted to add something about computer errors. In short, a filed bug can be designated as expected behavior for several reasons, and then adopted as part of the process.
This was an actual postmortem to code that managed account creation for credit report data. So very related to sign-in. The expected behavior was to continue account creation, even when the user failed their credit bureau verification. In doing so, there was a statistical subset where two accounts with two SSNs were linked.
Being assigned the bug, I fixed it by halting account creation on failing verification. This is my point, I made a value judgement and applied my own bias of what is right or wrong. The team lead drilled it into the team at the postmortem that there was no bug and errors had multiplied after my fix, further proof that the fix was incorrect.
In summary, there are real situations in which the computer's ability to be impartial is important. Is it really an error? To a business, ethical questions are debatable, sometimes what's important is to execute steps without bias.
The short answer is that we don't rely on them being error-free to any greater degree than we rely on anything else in a critical system to be error-free. Just as an individual computer hardware component can fail (and will fail in certain circumstances), any other non-computer component (mechanical, human or otherwise) can also fail. Just as the procedures ("programs") we write for computers can be incorrectly written, so can the procedures we write for humans to follow.
We achieve high reliability in systems not by counting on things not to fail but by knowing how and when they fail and designing the system to accommodate those failures.
Someone compared self driving cars to cars driven by humans. I would look at text messages instead: Delivering a text message without computer would force you to write it say on a postcard, take it to a letter box, where it is collected, sorted by delivery town, then some postman takes it to my home and drops it through the letter box.
Not only would the error rate be a lot higher (probably one percent or more of lost or misdelivered postcards), but you could expect a cost of say $0.50 per postcard, not counting the time it takes for you. There are kids sending hundreds of messages a day, they would bankrupt their parents in a short time.
A cheap computer can easily do 100 trillion floating-point operations in a day. So if that is what you need to do, then you have no choice but using a computer.
You get the wrong impression if you look at stack overflow for questions how to fix bugs. These bugs are 99% of the time not in shipping software! If an inexperienced developer creates a bug, either they detect immediately that their software doesn't work and fix it, or some more experienced developer checks it and it gets fixed, or QA tests the software, finds the bug, fixes it, or sometimes a bug appears only in very exceptional circumstances, so nobody ever finds it.
And you get the wrong impression again because humans make many mistakes that go undetected. If your car is repaired, but the repair is badly done, you will likely never find out. If your hotel bill, calculated by a human, is not correct, you will never find out. Computer programs do the same things again and again, and that is how mistakes get found out and fixed.
I can assure you that safety-critical software is produced in a way which ensures a high level of "software integrity".
These strategies make it possible to produce software that exposes so few catastrophic failures that it can be used where such failure would mean death.
The alternative is to not rely on computers. While there are some instances where humans clearly outperform computers (looking at you, Tesla Autopilot), in the vast majority of cases where computers do something safety-critical, they also outperform humans by a significant margin (c.f., anti-lock brakes, traction control systems, flight computers, pacemakers, etc. ad nauseum).
EDIT Since the Tesla fanboys are so easily triggered by the shade I threw at Autopilot, I'll just leave this here for your viewing pleasure...
