0
$\begingroup$

I read this paper obout simCLR and I didn't understand a few things:

  1. What is the goal of the temperature parameter τ? How does it affect the loss?
  2. What will be the value of the loss for a batch with 2N identical samples? Is there a mathematical formulation?
  3. In that case (question 2), will different temperature values change the loss?
  4. How does the batch size affect the contrastive loss? Is large batch size better than small batch size when training a model with contrastive loss?

Thanks!

Improve this question
$\endgroup$
3
  • $\begingroup$ Can you edit your post to provide a reference that fulfills the minimal scholarly requirements and will be robust over time? This helps others with a similar question find your post via web search. We have collected some advice here. Thank you! $\endgroup$ – D.W.♦ Jan 15 at 23:29
  • $\begingroup$ We generally recommend that you ask only one question per post. If you have multiple questions, they can be asked separately. $\endgroup$ – D.W.♦ Jan 15 at 23:30
  • $\begingroup$ I'm not sure what kind of answer you expect to your question 1. If you look at the formula for the loss it has $\tau$ so that equation describes quantitatively exactly how it affects the loss. I suggest you do a little research on temperature in softmax (you should be able to find a bunch of resources). $\endgroup$ – D.W.♦ Jan 15 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.