fossilesque@mander.xyzM to Science Memes@mander.xyzEnglish · 9 months agoMagic πmander.xyzimagemessage-square69fedilinkarrow-up1733arrow-down17
arrow-up1726arrow-down1imageMagic πmander.xyzfossilesque@mander.xyzM to Science Memes@mander.xyzEnglish · 9 months agomessage-square69fedilink
minus-squareAdkml [he/him]@hexbear.netlinkfedilinkEnglisharrow-up12arrow-down1·9 months ago“Only” using 15 digits is still pretty insane
minus-squareblind3rdeye@lemm.eelinkfedilinkEnglisharrow-up17·9 months agoYou get that level of precision in a standard “double” floating point number. So that’s basically the normal level of precision you get without trying.
minus-squareradiofreeval [any]@hexbear.netlinkfedilinkEnglisharrow-up9·9 months agoM_PI in math.h is like 20 digits. I’m surprised they just don’t do that.
minus-squareneptune@dmv.sociallinkfedilinkEnglisharrow-up4·9 months agoGoogle suggests that excel uses 15. So even college students working on any old STEM degree are probably using 15 digits.
“Only” using 15 digits is still pretty insane
You get that level of precision in a standard “
double” floating point number. So that’s basically the normal level of precision you get without trying.M_PIinmath.his like 20 digits. I’m surprised they just don’t do that.Google suggests that excel uses 15. So even college students working on any old STEM degree are probably using 15 digits.