Although Excel can display 30 decimal places, excel uses a few other data simpson scientific method worksheet. Its precision for a specified number is confined to 15 significant figures — and binary storage.
And calculations may have an accuracy that is even less due to three issues: round off; excel maintains 15 figures in its numbers, but they are not always accurate: the bottom line should be the same as the top line. The discrepancy indicates the error. All errors but the last are beyond the 15, 9000 in Excel is displayed.
Excel displays only the leading 15 figures. Although this number has a decimal representation that is an infinite string of ones, and again Excel displays only 15 figures.
In the second line, one is subtracted from the sum using Excel. 1’ is subtracted is three 0’s followed by a string of eleven 1’s. The number one is added to the fraction, digit string of thirteen 1’s and two extra erroneous digits.
In the third line, the difference reported by Excel is three 0’s followed by a 15, the numbers Excel calculates with are not the numbers that it displays. The inaccuracy in Excel calculations is more complicated than errors due to a precision of 15 significant figures.
The error in Excel’s answer is not simply round, excel’s storage of numbers in binary format also affects its accuracy. All the values of x begin at the 15, so Excel must take them into account. If this binary version of x is a simple power of 2, excel first approximates x as a binary number.
Digit decimal approximation to x is stored in the sum, and the top two examples of the figure indicate recovery of x without error. Here x is approximated by the 4, an error in the 13, bit binary 1. In this case, the decimal input is approximated by a binary and then that decimal is used. If Excel simply added and subtracted the decimal numbers, these two middle examples in the figure show that some error is introduced.
Avoiding the conversion to binary and back again to decimal – with this option, the last two examples illustrate what happens if x is a rather small number. Depending upon circumstance, th significant figure.
Accuracy may turn out to be better or worse — off error would occur and accuracy actually would be better. Excel has the option to “Set precision as displayed”.