0
$\begingroup$

A device is generating 14-bit integer which is stored as int16 (short), a scaling process will then scale the data to value of order 10E-3.

Does it then matter if I store these number with half float or single float? Suppose the scaling factors are just single float.

$\endgroup$
  • $\begingroup$ This is an exercise, right ? $\endgroup$ – TEMLIB Jul 26 '18 at 0:32
  • $\begingroup$ exercise? No, it's my situation: The data is from a digitizer which gives the 14-bit int data, and I am scaling them then storing them in float. At the moment I am using single float, but I wonder if I could just use half to make things faster for GPU processing. Sorry if it is something rather basic but this isn't my expertise. $\endgroup$ – Sandbo Jul 26 '18 at 1:16
  • $\begingroup$ Sorry. Standard IEEE 16bits "half float" has 11 bits mantissa. So, you will probably loose precision. Of course, if it is some analog measurement, maybe the lowest bits are not really significant (noise) and the input data not full range. And, if it your input is a signed integer, actual resolution is 13bits. $\endgroup$ – TEMLIB Jul 26 '18 at 19:43
  • $\begingroup$ Thanks for your input, I didn't know half is actually 11 bits mantissa (data is signed int), this clears a confusion for me. As they are indeed analog data so it may or may not really matter at the end if I lose 2 bit at the end. I will try to come up with some test signals to find out. $\endgroup$ – Sandbo Jul 27 '18 at 3:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.