All about floating point arithmetic for bedtime reading

[Retro]

Written in 1991, this will be of interest to programmers and fellow nerds who want to know more about floating point calculations on computers. It's all about the inevitable rounding error that happens with floating point calculations and how to handle it.

Devoid of pictures or any decent formatting, this is hardcore information on this subject. I dare say this will make excellent bedtime reading, curing insomnia in no time.

Nerd level: root 2.

Floating-point arithmetic is considered an esoteric subject by many people. This is rather surprising because floating-point is ubiquitous in computer systems. Almost every language has a floating-point datatype; computers from PCs to supercomputers have floating-point accelerators; most compilers will be called upon to compile floating-point algorithms from time to time; and virtually every operating system must respond to floating-point exceptions such as overflow. This paper presents a tutorial on those aspects of floating-point that have a direct impact on designers of computer systems. It begins with background on floating-point representation and rounding error, continues with a discussion of the IEEE floating-point standard, and concludes with numerous examples of how computer builders can better support floating-point.

What Every Computer Scientist Should Know About Floating-Point Arithmetic

docs.oracle.com

Tip: you might want to copy the text into Word or something to make the presentation more readable.

Here's a random flowing point picture from the internet to make up for the lack of article pictures and its source:

Optimal Architecture of Floating-Point Arithmetic for Neural Network Training Processors

The convergence of artificial intelligence (AI) is one of the critical technologies in the recent fourth industrial revolution. The AIoT (Artificial Intelligence Internet of Things) is expected to be a solution that aids rapid and secure data processing. While the success of AIoT demanded...

www.mdpi.com

 

[Arantor]

I have a lot of skepticism about AI, I have so much more skepticism about IoT, combining the two seems like a really bad idea.

This paper was interesting but I find it has some “really?” moments in it, such as “doing a bunch of calculations with x amount of data costs abc amount of energy, but if we make x bigger it consumes more energy” and this almost being somehow surprising given the arena they were working in. (I am not *entirely* convinced that they understood *why*, 24 bit ops are always going to be more expensive than 16 bit ops in general but it wasn’t clear if this was on optimised-for-16 or optimised-for-32 chips because on a properly designed optimised for 32 chip, even the extra data can be handled more efficiently than something half the size. But 24 is an outlier in general unless you expressly design for it. Powers of 2 are friendlier.)

The reality is that people aren’t going to be doing the actual crunching on IoT devices, it isn’t time efficient, it isn’t energy efficient and I don’t entirely think they’re solving the problem at the right level, though it’s interesting to watch someone take on IEEE 754.

 

[Retro]

Sorry Arantor, I think my post may have been a bit accidentally misleading. The article it's about is the link "What Every Computer Scientist Should Know About Floating-Point Arithmetic" that I've now made bold. It won't unfurl into a preview unfortunately, to be more obvious.

The pic was just to give us something to look at and I figured that linking to the article it came from was appropriate. I have no idea what was at that second article, lol. It can still be a valid subject for discussion though and you're points are most likely right.

 

[Arantor]

Oops, I didn’t even see the other link!

Yeah, I see what you mean about it being a bit dry, but honestly, IEEE 754 isn’t exactly joyous reading and (thankfully) most programmers have access to a library that does the hard work for them.

One thing to note, the relative majority of the world doing commercials will actually be using floating point, but in levels accurate enough for most things. Lots of internalised “rounding to two decimal places” shenanigans.

 

[Tiffany]

Impressive and interesting!

 

[Arantor]

It actually is when you stop and think about it - humans naturally think in terms of 10s both before and after the decimal point but people don't always stop to think about how the world looks when you're dealing in just 1s and 0s.