Justin Shenk
  • Member for 4 years, 4 months
  • Last seen more than a week ago
1 answers
4 votes
34 views
What is the primary reference for the observation/discussion of how neural networks struggle with ambiguous training datasets?
2 votes

Traditional single-label classification implies disjoint labels. To the extent that labels are not disjoint or distinct within a single-label classification task, the problem is one of the definition ...

View answer
4 answers
7 votes
2k views
Why is the manifold hypothesis true?
1 votes

The manifold hypothesis is perhaps best seen as a heuristic. The practical observation is that dimension reduction often works well, particularly with sufficiently dense samples. Intuitively, ...

View answer
1 answers
1 votes
58 views
Matrix-vector multiplication using only lower triangular of matrix
0 votes

Answer by Clayton Gotberg [1], modified: If $\textbf{A}$ is a symmetric matrix and $\textbf{A}_{LT}$ is the lower triangular part of the matrix and A_{UT} is the upper triangular part of the matrix: $...

View answer
2 answers
1 votes
24 views
Can we supervise on the hidden states of RNN?
Accepted answer
1 votes

There are several examples of performing supervision on hidden states for history dependent models. Latent variables Loss functions are commonly applied to internal states such as latent variables in ...

View answer
1 answers
1 votes
304 views
What is branch factor for beam search in RNNs like TensorFlow's Magenta?
1 votes

Branch factor in beam search refers to average number of branches at each level of a search tree. When predicting a sequence in an LSTM model, such as note selection in melody generation, $b$ branches ...

View answer
1 answers
2 votes
3k views
Beam size is a parameter in some RNNs like TensorFlow's Magenta. What is beam size?
2 votes

Beam size, or beam width, is a parameter in the beam search algorithm which determines how many of the best partial solutions to evaluate. In an LSTM model of melody generation, for example, beam size ...

View answer
1 answers
30 votes
23k views
What is Temperature in LSTM (and neural networks generally)?
Accepted answer
36 votes

Temperature is a hyperparameter of LSTMs (and neural networks generally) used to control the randomness of predictions by scaling the logits before applying softmax. For example, in TensorFlow’s ...

View answer