1
$\begingroup$

I am reading Long-term Temporal Convolutions for Action Recognition and under the Section 3.1, I read this:

To investigate the impact of long-term temporal convolutions, we here study network inputs with different temporal extents.....

and then

As illustrated in Figure 2, the temporal resolution in our 60f network corresponds to 60, 30, 15, 7 and 3 frames for each of the five convolutional layers. In comparison, the temporal resolution of the 16f network is reduced more drastically to 16, 8, 4, 2 and 1 frame at each convolutional layer. We believe that preserving the temporal resolution at higher convolutional layers should enable learning more complex temporal patterns. The space-time resolution for the outputs of the fifth convolutional layers is $3 \times 3 \times 1$ and $1 \times 1 \times 3$ for the 16f and 60f networks respectively. The two networks have a similar number of parameters in the fc6 layer and the same number of parameters in all other layers. For a systematic study of networks with different input resolutions we also evaluate the effect of increased temporal resolution $t \in \{20, 40, 60, 80, 100\}$ and varying spatial resolution of $\{58 \times 58, 71 \times 71\}$ pixels.

How can we reduce temporal resolution of a ConvNet at each convolutional layer.

What does preserving the temporal resolutional mean and what does complex temporal resolution mean?

I've taken courses on Convolutional Neural Network and never heard of anything such as "temporal extent" or "temporal resolutions". Searching about them gives links to other research paper, which still use the term without describing its meaning.

Anyone, kindly put some light on it.

$\endgroup$
6
  • $\begingroup$ It might be the same as duration. $\endgroup$ Feb 15, 2019 at 14:11
  • $\begingroup$ What kind of duration? Does it differs from the no. of frames per second? $\endgroup$ Feb 15, 2019 at 14:32
  • $\begingroup$ Number of frames per second is frequency. Duration is measured in units of time. For example, 1 second is a duration, while 20 frames per second is frequency. $\endgroup$ Feb 15, 2019 at 14:33
  • $\begingroup$ Aah !! That's right. Found this $\endgroup$ Feb 15, 2019 at 14:43
  • 2
    $\begingroup$ Feel free to answer your own question, if you think you have found a better answer. $\endgroup$
    – D.W.
    Feb 15, 2019 at 17:59

2 Answers 2

1
$\begingroup$

It's just the ordinary meaning of the words extent and resolution, only applied to time instead of space. You can think of space-time as a kind of a continuous block, instead of just thinking of time as something that ticks away.

E.g. if you imagine a graph where one axis is space, and another is time, then the spatial extent of something would the width of that thing on the space axis (that is, how much space it takes up). Spatial resolution is how tiny you can draw two distinct things on the graph before you can't tell them apart anymore. That's the technical definition - the smallest separation at which you can resolve two things of interest, although we tend to think of resolution as of the size of the image in pixels.

Similarly, a temporal extent is is the width of something on the time axis. How long it extends through time. So it can be a duration of something, or expressed in some other time-related unit - like how many frames it takes up. Temporal resolution is how small a separation in time can two tings have, before you can't tell that there is a separation between them (e.g., if two things happen rapidly at different times within the same frame, and all your data is at the frame level, you can't tell if the two things were separated or simultaneous, if their time spans overlap or not. So, a FPS rate gives you the temporal resolution (after you invert it to seconds per frame). If the neural network is doing something that bundles information from a number of consecutive frames together (maybe it processes them in blocks of several frames), as some information is discarded, the time-resolution might drop (for example, think of severe lag in videogames - one frame you were just walking, next frame it's game over and you have no idea what happened).

enter image description here

what does complex temporal resolution mean?

The term "complex temporal resolution" is not mentioned anywhere in the text. It says:

"We believe that preserving the temporal resolution at higher convolutional layers should enable learning more complex temporal patterns."

I'm paraphrasing based on a quick look of the paper, but basically, what they are saying is: they think that if they can preserve (after all the transformations or whatever they are doing) the original fidelity when it comes to distinguishing two moments in time (i.e., this capability doesn't get worse as information goes through the network), then their neural network would have a better chance of learning/recognizing things that happen over time (e.g. that someone is doing something in a video) - that are potentially more complicated than what was possible before (e.g. instead of just being able to recognize that someone is swimming, it could maybe recognize some more complex action).

New contributor
Filip Milovanović is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
1
  • $\begingroup$ Oh, damn, I just realized this is a question from 3 years ago XD. Never mind, hopefully it will be helpful to someone. $\endgroup$ yesterday
0
$\begingroup$

Temporal extent is the same as duration. It is measured in units of time, like second. For example, 1 second is a duration or temporal extent.

$\endgroup$
3
  • $\begingroup$ "Temporal extent refers to how much time a behavior takes up. For instance, if you are interested in measuring the behavior of crying, you can measure the duration of crying by starting a timer at the first sound of crying and ending the timer when the crying stops." source $\endgroup$ Feb 15, 2019 at 16:38
  • $\begingroup$ What does that mean in the context of videos with the input of 60 frame or 16 frames? $\endgroup$ Feb 15, 2019 at 17:06
  • $\begingroup$ It might refer to the duration of a single frame, or of a certain event. You'll have to read the paper and figure out from context. $\endgroup$ Feb 15, 2019 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.