Floating point (IEEE-754)

Q: Why is 0.1 not exact in floating point?

In binary, 0.1 is the repeating fraction 0.0001100110011... with no finite representation, just as 1/3 repeats forever in decimal. IEEE-754 has only a fixed number of mantissa bits, so it stores the nearest representable value, which is slightly off. That tiny error is why 0.1 + 0.2 does not equal exactly 0.3.

Q: What is the difference between single and double precision?

Single precision (float) uses 32 bits: 1 sign, 8 exponent, 23 mantissa, with an exponent bias of 127. Double precision uses 64 bits: 1 sign, 11 exponent, 52 mantissa, bias 1023. Double's extra exponent bits give a far wider range and its extra mantissa bits give much higher precision, at twice the storage and bandwidth.

Q: What is the hidden bit in IEEE-754?

A normalized binary significand always starts with a 1 (it is the only nonzero binary digit), so IEEE-754 does not store that leading 1; it is implied. The mantissa field holds only the fraction after the point, and the hardware prepends the hidden 1. This gives single precision 24 bits of effective precision from 23 stored bits. Subnormal numbers are the exception: they have no hidden 1.

Scientific notation in binary

9 min read

Floating point is a binary format, standardized as IEEE-754, that represents a real number as a sign, a biased exponent, and a fractional mantissa, trading exact even spacing for a huge dynamic range so the same fixed number of bits can hold both very large and very small values.

Builds onTwo's complement Binary numbers

Binary integers are exact, and two's complement gives them a sign, but they have a fixed range and no fractional part. A computer also has to handle 0.0000001 and 6.022 x 10^23, numbers spanning dozens of orders of magnitude, and no fixed-width integer can do that. The answer is floating point: store the number the way scientists write it, as a sign times a fraction times a power of two, and let the exponent slide the binary point wherever it needs to go. The standard that every modern CPU, GPU, and language agrees on is IEEE-754.

Fixed point versus floating point

One way to get fractions is fixed point: nail the binary point at a chosen place, say 8 integer bits and 8 fraction bits. It is simple and the spacing between values is uniform, but range and precision are locked together: those 16 bits can only reach about 256 and resolve to 1/256, and you cannot stretch to represent both an astronomical and a microscopic value. Floating point unlocks them by storing the position of the point as a separate exponent. You gain an enormous range; the price is that the spacing between representable values is no longer uniform, it widens as numbers get bigger.

Picture scientific notation: 6.022 x 10^23 is a sign (+), a significand (6.022), and an exponent (23). Floating point is exactly this, but in base 2: every number is +/- 1.fraction x 2^exponent. The 'floating' name is literal: the exponent decides where the binary point sits, so it floats left for tiny numbers and right for huge ones, instead of being pinned in one spot. See the exact sign, exponent, and mantissa of any value with the IEEE 754 converter.

The three fields: sign, exponent, mantissa

An IEEE-754 number packs three fields, left to right: a sign bit, a biased exponent, and a fractional mantissa (also called the significand). The sign is 0 for positive, 1 for negative. The value of a normal number is:

value = (-1)^sign x 1.mantissa x 2^(exponent - bias)

Format	Total bits	Sign	Exponent	Mantissa	Exp. bias
Single (float)	32	1	8	23	127
Double	64	1	11	52	1023

The two common formats. Single precision is 1 + 8 + 23 bits; double is 1 + 11 + 52. More exponent bits buy range; more mantissa bits buy precision. The bias is subtracted from the stored exponent to get the real one.

The biased exponent

The exponent needs to be signed (numbers smaller than 1 need a negative exponent), but IEEE-754 does not store it in two's complement. Instead it uses a bias: the stored field is the real exponent plus a fixed offset (127 for single). So a stored 127 means exponent 0, stored 128 means +1, stored 126 means -1. Why bias rather than two's complement? Because then the bit patterns sort in the same order as the numbers: a larger float has a larger stored exponent field, so hardware can compare floats with an ordinary unsigned integer comparator, no special sign handling. The all-zero and all-one exponent fields are reserved (see special values), so a single's usable stored range is 1 to 254, giving real exponents -126 to +127.

Normalized form and the hidden 1

In decimal scientific notation you normalize so one nonzero digit sits before the point (6.022, not 60.22). In binary the only nonzero digit is 1, so a normalized binary significand is *always* 1.something. Since that leading digit is guaranteed to be 1, IEEE-754 does not waste a bit storing it: the mantissa field holds only the fraction *after* the point, and the hardware silently prepends the hidden (implicit) 1. That free bit is why single precision has 23 stored mantissa bits but 24 bits of real precision.

Worked example: encode -6.5 as a 32-bit float

Sign. The number is negative, so the sign bit is 1.
Magnitude in binary. 6.5 = 4 + 2 + 0.5 = 110.1 in binary.
Normalize. Shift the point so one 1 is in front: 110.1 = 1.101 x 2^2. The mantissa fraction is 101, the exponent is 2.
Bias the exponent. Stored exponent = 2 + 127 = 129 = 1000 0001.
Pad the mantissa to 23 bits. 101 followed by twenty zeros: 101 0000 0000 0000 0000 0000.
Assemble sign | exponent | mantissa: 1 10000001 10100000000000000000000, which groups into the hex 0xC0D00000.

Decoding runs the same steps backward. Take 0 10000010 01000000000000000000000: the sign is 0 (positive), the stored exponent 10000010 = 130 gives a real exponent 130 - 127 = 3, and the significand is the hidden 1 plus .01, that is 1.01 = 1.25. So the value is +1.25 x 2^3 = +10.0.

Special values

The reserved all-zero and all-one exponent fields encode the cases plain normalized numbers cannot. With the exponent all zeros there is no hidden 1: an all-zero mantissa is zero (and because the sign bit is separate, there is a +0 and a -0), while a nonzero mantissa is a subnormal, a very small number using 0.mantissa x 2^(1 - bias) to fill the gap between the smallest normal number and zero (gradual underflow). With the exponent all ones, an all-zero mantissa is infinity (+inf or -inf, from overflow or divide-by-zero), and a nonzero mantissa is NaN ('not a number'), the result of 0/0, inf - inf, or sqrt(-1).

Exponent field	Mantissa	Represents
all 0	0	+/- zero (signed)
all 0	nonzero	subnormal number
1 .. 254 (normal)	any	+/- 1.mantissa x 2^(e - 127)
all 1	0	+/- infinity
all 1	nonzero	NaN

How the exponent field selects the kind of value (single precision shown). The two extreme exponent patterns are reserved, which is why the usable normal range stops at stored 254 and starts at stored 1.

Rounding, ULP, and why 0.1 is not exact

Only numbers of the form 1.fraction x 2^e with a short enough fraction are representable, so most reals must be rounded to the nearest one. The default rule is round-to-nearest-even: on a tie, round so the last mantissa bit is 0, which avoids the slow drift a 'round half up' rule would introduce. The classic surprise is 0.1: in binary it is 0.0001100110011... repeating forever, exactly the way 1/3 repeats in decimal, so it cannot be stored exactly. The stored value is a hair off, which is why 0.1 + 0.2 does not give exactly 0.3 in floating point.

The gap between two consecutive representable numbers is the ULP (unit in the last place). Because the exponent scales the whole significand, the ULP grows with magnitude: near 1.0 the gap in single precision is about 0.0000001, but for large numbers the gap can exceed 1, so a big enough float cannot even represent every integer. Floating point spends its precision where the numbers are small and spreads it thin where they are large.

Common mistakes. Never test floats for exact equality with ==; compare within a small tolerance instead, because rounding means the bits rarely match exactly. Do not assume arithmetic is associative: (a + b) + c can differ from a + (b + c) once rounding enters. The hidden leading 1 exists only for normal numbers (exponent 1 to 254); subnormals have no hidden 1. And remember the exponent is stored biased, not in two's complement, so reading it as a signed integer gives the wrong value.

IEEE-754 matters because it is universal: the same bit pattern means the same number, and the same operation rounds the same way, on essentially every processor and language, so numerical results are portable and reproducible. Understanding the three fields explains where floating-point error comes from, why money is better kept in integer cents or a decimal type than in binary floats, and how a designer trades range against precision by choosing single versus double.

A cautionary tale. In 1994 Intel's first Pentium shipped with the FDIV bug: a few missing entries in the floating-point divider's lookup table made certain rare divisions wrong from about the fifth significant figure, so a calculation like 4195835 / 3145727 returned a visibly incorrect answer. Correcting it in silicon cost Intel about $475 million to replace the affected chips. That is the lesson behind the standard: a processor must conform to IEEE-754 exactly and round every operation correctly, because 'almost right' arithmetic, multiplied across millions of shipped chips, is ruinously expensive to fix.

Try it

Encode +0.75 into 32-bit IEEE-754 single precision. Give the sign bit, the real (unbiased) exponent, the biased exponent field in binary, and the leading mantissa bits.

Frequently asked

What is floating point in computing?

Floating point is a way to store real numbers as a sign, an exponent, and a fractional mantissa, so a fixed number of bits can represent a very wide range of magnitudes. The standard format, IEEE-754, encodes a normal number as (-1)^sign x 1.mantissa x 2^(exponent - bias), the binary equivalent of scientific notation.

Why is 0.1 not exact in floating point?

In binary, 0.1 is the repeating fraction 0.0001100110011... with no finite representation, just as 1/3 repeats forever in decimal. IEEE-754 has only a fixed number of mantissa bits, so it stores the nearest representable value, which is slightly off. That tiny error is why 0.1 + 0.2 does not equal exactly 0.3.

What is the difference between single and double precision?

Single precision (float) uses 32 bits: 1 sign, 8 exponent, 23 mantissa, with an exponent bias of 127. Double precision uses 64 bits: 1 sign, 11 exponent, 52 mantissa, bias 1023. Double's extra exponent bits give a far wider range and its extra mantissa bits give much higher precision, at twice the storage and bandwidth.

Why is the floating-point exponent biased instead of two's complement?

Biasing (storing the real exponent plus an offset, 127 for single) makes the exponent field, and therefore the whole number, sortable as an ordinary unsigned integer: a bigger float has a bigger stored pattern. That lets hardware compare floats with a plain integer comparator instead of decoding a two's-complement sign first.

What is the hidden bit in IEEE-754?

A normalized binary significand always starts with a 1 (it is the only nonzero binary digit), so IEEE-754 does not store that leading 1; it is implied. The mantissa field holds only the fraction after the point, and the hardware prepends the hidden 1. This gives single precision 24 bits of effective precision from 23 stored bits. Subnormal numbers are the exception: they have no hidden 1.

Floating point completes the tour of how computers represent numbers. Doing arithmetic on these fields (aligning exponents, adding the mantissas, then normalizing and rounding) is the job of a floating-point unit, a more elaborate relative of the integer ALU you build later in the course out of the very same adders and gates.

Every lesson here builds toward one thing: a working CPU, from the transistor up.

Open the free lab →