Questions tagged [floating-point]
Floating-point numbers are approximations of real numbers that can represent larger ranges than integers but use the same amount of memory, at the cost of lower precision. If your question is about small arithmetic errors (e.g. why does 0.1 + 0.2 equal 0.300000001?) or decimal conversion errors, please read the tag page before posting.
floating-point
15,306
questions
3967
votes
36
answers
587k
views
Is floating-point math broken?
Consider the following code:
0.1 + 0.2 == 0.3 -> false
0.1 + 0.2 -> 0.30000000000000004
Why do these inaccuracies happen?
- 45.2k
2736
votes
34
answers
4.8m
views
How do I parse a string to a float or int?
How can I convert an str to a float?
"545.2222" -> 545.2222
Or an str to a int?
"31" -> 31
For the reverse, see Convert integer to string in Python and Converting a float ...
- 68.6k
2456
votes
18
answers
1.3m
views
Difference between decimal, float and double in .NET?
What is the difference between decimal, float and double in .NET?
When would someone use one of these?
Tom
2409
votes
36
answers
5.7m
views
Limiting floats to two decimal points
I want a to be rounded to 13.95. I tried using round, but I get:
>>> a
13.949999999999999
>>> round(a, 2)
13.949999999999999
For the analogous issue with the standard library ...
kevin
2314
votes
12
answers
236k
views
Why doesn't GCC optimize a*a*a*a*a*a to (a*a*a)*(a*a*a)?
I am doing some numerical optimization on a scientific application. One thing I noticed is that GCC will optimize the call pow(a,2) by compiling it into a*a, but the call pow(a,6) is not optimized and ...
- 24.7k
1987
votes
41
answers
1.8m
views
How do I check if a string represents a number (float or int)?
How do I check if a string represents a numeric value in Python?
def is_number(s):
try:
float(s)
return True
except ValueError:
return False
The above works, but it ...
- 20.3k
1673
votes
7
answers
164k
views
Why does changing 0.1f to 0 slow down performance by 10x?
Why does this bit of code,
const float x[16] = { 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8,
1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6};
const float z[...
- 14.9k
1267
votes
39
answers
2.0m
views
Format number to always show 2 decimal places
I would like to format my numbers to always display 2 decimal places, rounding where applicable.
Examples:
number display
------ -------
1 1.00
1.341 1.34
1.345 1.35
I ...
- 16.8k
1241
votes
16
answers
439k
views
Why not use Double or Float to represent currency?
I've always been told never to represent money with double or float types, and this time I pose the question to you: why?
I'm sure there is a very good reason, I simply do not know what it is.
- 18.3k
1177
votes
13
answers
1.2m
views
Is there a float input type in HTML5?
According to html5.org, the "number" input type's "value attribute, if specified and not empty, must have a value that is a valid floating point number."
Yet it is simply (in the latest version of ...
1014
votes
35
answers
1.2m
views
How do I use a decimal step value for range()?
How do I iterate between 0 and 1 by a step of 0.1?
This says that the step argument cannot be zero:
for i in range(0, 1, 0.1):
print(i)
- 101k
832
votes
46
answers
734k
views
How can I deal with floating point number precision in JavaScript? [duplicate]
I have the following dummy test script:
function test() {
var x = 0.1 * 0.2;
document.write(x);
}
test();
This will print the result 0.020000000000000004 while it should just print ...
- 32.7k
815
votes
30
answers
1.6m
views
How do you round UP a number?
How does one round a number UP in Python?
I tried round(number) but it rounds the number down. Here is an example:
round(2.3) = 2.0
and not 3, as I would like.
Then I tried int(number + .5) but it ...
- 78k
809
votes
13
answers
78k
views
How to convert Decimal to Double in C#?
I want to assign the decimal variable "trans" to the double variable "this.Opacity".
decimal trans = trackBar1.Value / 5000;
this.Opacity = trans;
When I build the app it gives ...
793
votes
11
answers
777k
views
How can I force division to be floating point? Division keeps rounding down to 0?
I have two integer values a and b, but I need their ratio in floating point. I know that a < b and I want to calculate a / b, so if I use integer division I'll always get 0 with a remainder of a.
...
- 126k
686
votes
7
answers
795k
views
How to get a random number between a float range?
random.randrange(start, stop) only takes integer arguments. So how would I get a random number between two float values?
- 7,585
664
votes
32
answers
661k
views
How do you compare float and double while accounting for precision loss?
What would be the most efficient way to compare two double or two float values?
Simply doing this is not correct:
bool CompareDoubles1 (double A, double B)
{
return A == B;
}
But something like:
...
- 6,681
620
votes
29
answers
851k
views
How to nicely format floating numbers to string without unnecessary decimal 0's
A 64-bit double can represent integer +/- 253 exactly.
Given this fact, I choose to use a double type as a single type for all my types, since my largest integer is an unsigned 32-bit number.
But now ...
- 27.9k
599
votes
5
answers
1.5m
views
Correct format specifier for double in printf
What is the correct format specifier for double in printf? Is it %f or is it %lf? I believe it's %f, but I am not sure.
Code sample
#include <stdio.h>
int main()
{
double d = 1.4;
printf(&...
- 6,001
593
votes
5
answers
79k
views
Why does Math.round(0.49999999999999994) return 1?
In the following program you can see that each value slightly less than .5 is rounded down, except for 0.5.
for (int i = 10; i >= 0; i--) {
long l = Double.doubleToLongBits(i + 0.5);
...
- 531k
541
votes
18
answers
462k
views
How to compare floats for almost-equality in Python?
It's well known that comparing floats for equality is a little fiddly due to rounding and precision issues. For examples on this, see the blog post Comparing Floating Point Numbers, 2012 Edition by ...
- 11.6k
539
votes
14
answers
1.2m
views
What is the difference between float and double?
I've read about the difference between double precision and single precision. However, in most cases, float and double seem to be interchangeable, i.e. using one or the other does not seem to affect ...
- 7,863
532
votes
17
answers
938k
views
How to parse float with two decimal places in javascript?
I have the following code. I would like to have it such that if price_result equals an integer, let's say 10, then I would like to add two decimal places. So 10 would be 10.00.
Or if it equals 10.6 ...
- 11k
505
votes
14
answers
619k
views
JavaScript displaying a float to 2 decimal places
I wanted to display a number to 2 decimal places.
I thought I could use toPrecision(2) in JavaScript .
However, if the number is 0.05, I get 0.0500. I'd rather it stay the same.
See it on JSbin.
...
- 487k
503
votes
16
answers
600k
views
How to format a float in javascript?
In JavaScript, when converting from a float to a string, how can I get just 2 digits after the decimal point? For example, 0.34 instead of 0.3445434.
F40
452
votes
16
answers
709k
views
How do I print a double value with full precision using cout?
In my earlier question I was printing a double using cout that got rounded when I wasn't expecting it. How can I make cout print a double using full precision?
427
votes
9
answers
859k
views
How to convert float to int with Java
I used the following line to convert float to int, but it's not as accurate as I'd like:
float a=8.61f;
int b;
b=(int)a;
The result is : 8 (It should be 9)
When a = -7.65f, the result is : -7 (...
- 30.9k
426
votes
8
answers
478k
views
Double vs. BigDecimal?
I have to calculate some floating point variables and my colleague suggest me to use BigDecimal instead of double since it will be more precise. But I want to know what it is and how to make most out ...
- 10.8k
422
votes
12
answers
101k
views
How dangerous is it to compare floating point values?
I know UIKit uses CGFloat because of the resolution independent coordinate system.
But every time I want to check if for example frame.origin.x is 0 it makes me feel sick:
if (theView.frame.origin.x ...
- 40.4k
388
votes
11
answers
1.0m
views
Convert floats to ints in Pandas?
I've been working with data imported from a CSV. Pandas changed some columns to float, so now the numbers in these columns get displayed as floating points! However, I need them to be displayed as ...
- 5,537
366
votes
32
answers
519k
views
Get decimal portion of a number with JavaScript
I have float numbers like 3.2 and 1.6.
I need to separate the number into the integer and decimal part. For example, a value of 3.2 would be split into two numbers, i.e. 3 and 0.2
Getting the integer ...
- 3,671
363
votes
12
answers
77k
views
What is the rationale for all comparisons returning false for IEEE754 NaN values?
Why do comparisons of NaN values behave differently from all other values?
That is, all comparisons with the operators ==, <=, >=, <, > where one or both values is NaN returns false, contrary to ...
- 56.2k
361
votes
25
answers
512k
views
How do I use floating-point arithmetic in bash?
I am trying to divide two image widths in a Bash script, but bash gives me 0 as the result:
RESULT=$(($IMG_WIDTH/$IMG2_WIDTH))
I did study the Bash guide and I know I should use bc, in all examples ...
- 4,843
358
votes
24
answers
399k
views
Python JSON serialize a Decimal object
I have a Decimal('3.9') as part of an object, and wish to encode this to a JSON string which should look like {'x': 3.9}. I don't care about precision on the client side, so a float is fine.
Is there ...
- 6,888
331
votes
11
answers
303k
views
What MySQL data type should be used for Latitude/Longitude with 8 decimal places?
I'm working with map data, and the Latitude/Longitude extends to 8 decimal places. For example:
Latitude 40.71727401
Longitude -74.00898606
I saw in the Google document
which uses:
lat FLOAT( 10, ...
- 9,668
329
votes
17
answers
586k
views
Random float number generation
How do I generate random floats in C++?
I thought I could take the integer rand and divide it by something, would that be adequate enough?
- 165k
323
votes
3
answers
167k
views
Float vs Decimal in ActiveRecord
Sometimes, Activerecord data types confuse me. Err, often. One of my eternal questions is, for a given case,
Should I use :decimal or :float?
I've often come across this link, ActiveRecord: :...
- 19k
322
votes
9
answers
576k
views
How to convert string into float in JavaScript?
I am trying to parse two values from a datagrid. The fields are numeric, and when they have a comma (ex. 554,20), I can't get the numbers after the comma. I've tried parseInt and parseFloat. How can I ...
F40
316
votes
22
answers
132k
views
Why can't decimal numbers be represented exactly in binary?
There have been several questions posted to SO about floating-point representation. For example, the decimal number 0.1 doesn't have an exact binary representation, so it's dangerous to use the == ...
- 20.5k
314
votes
2
answers
22k
views
What does the constant 0.0039215689 represent?
I keep seeing this constant pop up in various graphics header files
0.0039215689
It seems to have something to do with color maybe?
Here is the first hit on Google:
void RDP_G_SETFOGCOLOR(void)
{
...
- 17k
312
votes
8
answers
16k
views
Why does changing the sum order returns a different result?
Why does changing the sum order returns a different result?
23.53 + 5.88 + 17.64 = 47.05
23.53 + 17.64 + 5.88 = 47.050000000000004
Both Java and JavaScript return the same results.
I understand ...
- 13.7k
311
votes
7
answers
217k
views
Why is division in Ruby returning an integer instead of decimal value?
For example:
9 / 5 #=> 1
but I expected 1.8. How can I get the correct decimal (non-integer) result? Why is it returning 1 at all?
- 5,121
311
votes
7
answers
69k
views
Why are these numbers not equal?
The following code is obviously wrong. What's the problem?
i <- 0.1
i <- i + 0.05
i
## [1] 0.15
if(i==0.15) cat("i equals 0.15") else cat("i does not equal 0.15")
## i does not equal 0.15
- 5,373
310
votes
15
answers
419k
views
How to check if a float value is a whole number
I am trying to find the largest cube root that is a whole number, that is less than 12,000.
processing = True
n = 12000
while processing:
n -= 1
if n ** (1/3) == #checks to see if this has ...
- 13.1k
307
votes
11
answers
306k
views
Biggest integer that can be stored in a double
What is the biggest "no-floating" integer that can be stored in an IEEE 754 double type without losing precision?
In other words, at would the follow code fragment return:
UInt64 i = 0;
...
- 27.7k
306
votes
3
answers
27k
views
Why does NaN - NaN == 0.0 with the Intel C++ Compiler?
It is well-known that NaNs propagate in arithmetic, but I couldn't find any demonstrations, so I wrote a small test:
#include <limits>
#include <cstdio>
int main(int argc, char* argv[]) {...
- 15.1k
304
votes
13
answers
281k
views
Make a float only show two decimal places
I have the value 25.00 in a float, but when I print it on screen it is 25.0000000.
How can I display the value with only two decimal places?
ron
302
votes
12
answers
68k
views
When should I use double instead of decimal?
I can name three advantages to using double (or float) instead of decimal:
Uses less memory.
Faster because floating point math operations are natively supported by processors.
Can represent a larger ...
- 49k
297
votes
19
answers
214k
views
Formatting floats without trailing zeros
How can I format a float so that it doesn't contain trailing zeros? In other words, I want the resulting string to be as short as possible.
For example:
3 -> "3"
3. -> "3"
3.0 -> "3"
3.1 -&...
- 3,519
292
votes
10
answers
625k
views
Formatting a float to 2 decimal places
I am currently building a sales module for a clients website. So far I have got the sale price to calculate perfectly but where I have come stuck is formatting the output to 2 decimal places.
I am ...
- 3,341