Suppose you have a program which works on a stream of images. It gets an image, makes edits and stores the image. What is the maximum latency with which one can still speak of a "real time application"? (Please give a source in your answer)
-
$\begingroup$ The informal definition of "real time" is "an answer which is too late is a wrong answer". What constitutes "too late" is context-dependent. A system which processes monthly payrolls is useless if it takes three months to run, for example. Almost every system has real-time constraints, but not all of them need to be mentioned explicitly. $\endgroup$– Pseudonym ♦Jan 25, 2017 at 0:32
3 Answers
There is no one answer to this question. "Real time" can mean very different things, depending on what kind of sensor or sensors you are using, and what you are trying to do. By the way, editing and storing images may not even qualify as computer vision. Computer vision is about extracting information from images or video, and the meaning of real time depends on the kind of information you are trying to extract.
A typical video camera acquires images at 30 frames per second (fps). So that can be your definition of "real-time", i.e. being able to process images at the rate at which you acquire them.
On the other hand, if your goal is to detect and track objects in videos, then "real-time" is a function of how fast those objects move. For example, 10fps is often sufficient for tracking people walking on the street, but probably not for tracking cars going 70 miles per hour on a highway.
In computer science, a real-time computing (RTC) system is one that guarantees a response before a previously set deadline. For a stream of images, that implies that a real-time system is one that guarantees it has finished processing each image before some previously set timing constraint on latency.
There is no 'maximum latency with which one can still speak of a "real time application"'. Each real-time system has its own fixed maximum latency, that generally differs from the maximum latency of other real-time systems.
RealTime is simply about meeting timing constraints. Those timing constraints might be microseconds, or might be hours. -- http://wiki.c2.com/?NearRealTime
and
EmbeddedSystems engineers use exactly one definition of real time, and that is this: A real time system employs tasks which execute to completion in a guaranteed amount of time. -- http://wiki.c2.com/?RealTime
and
real-time systems... have to provide results which are not only correct, but also delivered in time. Instead of average behavior as for standard computing, real-time systems have to allow for guarantees that the temporal requirements will be met. -- http://www.ecrts.org/
and
Real-Time System: is a computing system whose correct behavior depends not only on the value of the computation but also on the time at which outputs are produced. -- http://sites.ieee.org/tcrts/education/terminology-and-notation/
and
"In soft real time applications, system can miss the dead line without causing failure of the overall system. As an example ... losing a video frame will not fail ..." -- p. 356 "Computer-Based Industrial Control"
and
"Common miconceptions: ... Real-time computing is equivalent to fast computing? ... Rather than being fast ... the most important property of a real-time system should be predictability. ... Predictability, not speed, is the foremost goal in real-time-system design." -- John Stankovic. "Misconceptions about Real-Time Computing: A Serious Problem for Next-Generation Systems". 1988.
If you have a subsystem that can only handle one frame of video at a time, and video is streaming in at 30 frames per second and cannot be buffered, then to meet real-time constraints that subsystem must handle each frame in 1/30 of a second or less to qualify as a real-time system.
However, often there is a pipeline of such subsystems, each one doing its job in 1/30 of a second or less, but the total end-to-end latency from a frame of video introduced into one end to the results at the other end is a full second or more, and yet such a pipeline still qualifies as real-time when it is guaranteed to meet all its timing constraints.
-
$\begingroup$ You give a good notion of what is meant by a 'real-time computing system' in general, but that's not the same as what is meant by 'real time' in the computer vision context -- it's a name collision. (At least, that's my undertsanding.) I don't think the last paragraph is an accurate description of what computer vision folks mean when they refer to a 'real time' computer vision system. $\endgroup$– D.W. ♦Jan 24, 2017 at 21:01
-
$\begingroup$ @D.W. : You may be right. Feel free to post a more accurate answer. $\endgroup$ Jan 25, 2017 at 0:32
-
$\begingroup$ Upvoted as this is also my understanding of "real-time" as it is also defined in ISO/IEC 2382 generally for the "processing of data ...according to time requirements imposed by the outside process" iso.org/obp/ui/#iso:std:iso-iec:2382:ed-1:v1:en . Keep in mind, that it is NOT defined if that time-requirement is a millisecond, a second or even a year. $\endgroup$ Dec 11, 2017 at 11:57
The definition of Real-Time is usually dependent on some requirement for your system. Some systems might deem Real-Time as 10Hz, some might think 1Hz, etc. However, the rate at which your system needs to operate is often the upper bound of how fast some algorithms, Computer Vision or something else, need to run. So if your system needs to operate at 15Hz, you should expect your Computer Vision (or other) algorithms need to be faster than 15 Hz.