I have been coding and testing Neural Networks for a while but as of now I have only used IMAGE datasets. (i.e. I have M training images and N testing images).

Some datasets are video datasets. The UCSD Peds dataset for example has the following variants:

Peds 1: 34 training videos (of 200 frames each) and 36 testing videos (of 200 frames each) Peds 2: 16 training videos (of varying number of frames) and 12 testing videos (of varying number of frames)

So basically, in this case, I have M SETS of training images and N SETS of testing images.

I know the inputs to a neural network is arbitrary and anything can be fed, but I do not really understand how to feed a set of videos.

Will we merge all the 34 x 200 = 6800 training frames together and 36 x 200 = 7200 testing frames together and use the resulting set just like we use MNIST, etc?

How do we feed a set of training videos to a neural network?

What if I want to detect anomalies in the test videos? For example a neural network is trained using videos from UCSD dataset involving only pedestrians. And tested on videos which also have cars and bikes. So these cars and bikes are anomalies.

I'd want to classify an entire video on whether it contains just normal elements or some anomalous elements as well.

  • 1
    $\begingroup$ It's not clear what problem you are trying to solve, or what you are trying to train your neural network to do. $\endgroup$
    – D.W.
    Feb 14 '18 at 0:30
  • $\begingroup$ I don't think this is the best site to ask this on, since it seems to concern rather practical matter of how to implement NNs. Migrate to Computational Science or Artificial Intelligence? (cc @D.W.) $\endgroup$
    – Raphael
    Feb 14 '18 at 11:50
  • $\begingroup$ @Raphael, probably Data Science.SE or Stats.SE are the two I'd guess as most relevant. I wouldn't recommend AI as they seem to focus more on conceptual/philosophical rather than technical implementation/engineering. Not sure Computational Science sees much of this sort of question. In any case, I'm somewhat reluctant to migrate until I understand better what the question is. $\endgroup$
    – D.W.
    Feb 14 '18 at 16:30
  • $\begingroup$ Just made it more clear. I hope you understand the question now? $\endgroup$
    – user83728
    Feb 14 '18 at 17:11
  • $\begingroup$ Thanks for making an edit to elaborate! I guess my confusion is about something different. My confusion is what you're trying to accomplish with your classifier. Suppose you figure out how to train a classifier, and it is great. How will you use it? Are you going to use it to classify something about a single frame of video? Or are you going to use it to classify something about an entire video? You'd train it one way for the former, and a different way for the latter. The method to use will depend on what you are trying to achieve. $\endgroup$
    – D.W.
    Feb 14 '18 at 17:25

If you want to detect anomalies, my suggestion would be to build a classifier to classify individual frames of the video to determine whether that frame contains an anomaly. You'll need to categorize what you count as an "anomaly" and arrange for labelled data in your training set. But then you can extract all 6800 frames from the training videos, label each frame, and train a classifier on that training set of 6800 images. At this point the problem is the same as building a classifier on individual images. Then, given a test video, you apply the classifier to each frame to determine whether any of its frames contain an anomaly, and aggregate those results somehow (e.g., report the video as anomalous if the number of anomalous frames exceeds some threshold).

  • $\begingroup$ +1 ed. So basically feeding a video dataset is the same as merging all the frames of all videos together and feeding them image by image. There seems to be no other sensible technique right? $\endgroup$
    – user83728
    Feb 15 '18 at 1:49
  • $\begingroup$ @Cosmonavt, there are other techniques but I am suggesting one that is probably most appropriate for the particular situation you asked about. $\endgroup$
    – D.W.
    Feb 15 '18 at 6:36
  • $\begingroup$ Yes what are the other techniques? Can you elaborate a bit? $\endgroup$
    – user83728
    Feb 15 '18 at 12:21
  • 1
    $\begingroup$ You could alternatively use Conv3D or ConvLSTM2D. $\endgroup$
    – Lugi
    Feb 15 '18 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy