## Can the implicit conversion by the FORMAT function of Lisp be controlled?

While trying to round a fraction to 9 digits, I came across the following: ? (format t "~9\$" 763700091/112000148) 6.818742300 Nil But the rounding is only accurate to 5 digits. I expected format would take care of this automatically; ie, convert

## Get String Zero Decimal Floating Point

I have a string string(5) "388.0". I simply want to get exactly float number with one zero or two zero or whatever numbers comes after decimal. How can I achieve it? Simply (float)\$val gives me only float(388) without zero.Have you tried the bui

## Is there a difference between -x vs x * -1.0f?

In java, is there a difference between negating a variable vs multiplying a variable by a float minus one, if the variable is also a float?In JLS §15.15.4 "Unary Minus Operator -", we find that For floating-point values, negation is not the same

## Why does Python issue an error when I read a string from a particular column of a csv file and try to parse it into a floating point number?

I am using the following python code to read numbers out of the csv file: with open('file.csv') as p: rows = csv.reader(p) for row in rows: numbers = float(row[3]) I understand that although the csv file contains numbers, they are read as strings. In

## How to filter floats with integers [logic]

I have an array of floats. Some of these floats are VERY close to integers, but they're still floats. I need to filter these values. For example: array_n = [n]; //an array of floats k = 160. In the if loop, n is an element in array_n. I need to pick

## ARM Cortex M4 Hardware Error - Floating Point

When I run my program, which just calculates a sine wave: for(i = 0; i < ADS1299_SIGNAL_WINDOW; i++){ TEST[i] = (float32_t)(10.0f * (float32_t)(arm_sin_f32((float32_t)(3.14f * i/ADS1299_SIGNAL_WINDOW)))); } The compiler generates the following line,

## Best polynomial minimax approximation optimized by arctangent machine on [-1,1]?

For the simple and efficient implementation of fast math functions with reasonable accuracy, polynomial minimax approximations are often the method of choice. Minimax approximations are typically generated with a variant of the Remez algorithm. Vario

## Python vs division negative exhibitor

In Python, should I use negative exponents or division? Example: >>> num = 2.0**-1 or >>> num = 1/2.0 I ran a little test, and it looks like the difference comes down to BINARY_POWER vs. BINARY_DIVIDE: import dis def exp_test(x): return

## Divide two integers to get a float in MySQL

I'm working on a user defined function in MySQL that takes an integer (mediumint) and divides it by 100 (i.e. 10785 / 100 = 107.85). The supplied integer is a parameter called time. I've tried DECLARE dtime FLOAT; SET dtime = time / 100; DECLARE ftim

## The shortest way to check if Double is & ldquo; NaN & quot;

When calling Double.IsNaN() with Double.PositiveInfinity as argument, the result is false. This is against my intuition since infinity is not a number. Apparently "NaN" only exists in terms of a constant in .NET, is this described by the IEEE st

## Do Variables Contain Additional Hidden Metadata - aka When is zero not zero (but still is)

I hate having to ask this because I assume the answer must be simple, but I cannot for the life of me seem to track down the source. While trying to rewrite a function I ran across this problem: a = -j x = real(a) y = imag(a) y/x Which spits out Inf,

## strange arithmetic with swi-prolog

This question already has an answer here: Is floating point math broken? 22 answers I find the result very strange. Why not 0.3? Can somebody tell me why this result? Is it possible to fix this. ?- X is 5.3-5. X = 0.2999999999999998. ?- My second que

## Why is the float double, imprecise, double so as not to float?

I know that floating point variables loose precision while casting. But what I don't understand is, why a cast from a smaller primitive to a bigger one is unprecise but vice versa not. I would understand if it happens from double to float but it's th

## Is there a way to improve the floating point precision of this expression?

I have an expression that is used to estimate percentiles by interpolating between two values. windowMin + (currentPercentile - lastPercentile) * (windowMax - windowMin) / (percentile - lastPercentile) This has given me very good real-world results.

## Floating point arithmetic rounded to nearest

I am a little confused about rounding to nearest in floating point arithmetic. Let a, b, and c be normalized double precision floating point numbers. Is it true that a+b=b+a where + is correctly rounded to nearest floating point addition? My initial

## Configure GCC for soft-fp with exceptions on ARM?

According to this page on the GCC Wiki: http://gcc.gnu.org/wiki/Software_floating_point the "soft-fp" soft float implementation gcc provides has optional support for exceptions and rounding modes. Is there any way to configure GCC (libgcc) when

## Is float16 supported in matlab?

Does MATLAB support float16 operations? If so, how to convert a double matrix to float16? I am doing an arithmetic operation on a large matrix where 16-bit floating representation is sufficient for my representation. Representing by a double datatype

## .6 x 100 giving the result as 60.00004 instead of 60

Possible Duplicate: Mysterious calculation error when multiply by 100 Here is a strange problem that i am facing. I am i store a value .6 to a float variable. When i multiply it with 100 I am getting wrong answer as 60.000004.Where does this .000004

## How can I solve these problems with numbers?

I still have troubles with numbers in Vim: p.e. let a = 1.02 | let b = '10000000' | let total = a*b | echo total --> 1.02e7 I would like to avoid Exponentials. How can I have the output 10200000 ? let a = 4000000000 | let b = '1' | let total = a+b |

## Java Floating Point Math Error?

I am trying to calculate an aspect ratio for an image: Log.d("img.width:", String.valueOf(img.getIntrinsicWidth())); Log.d("img.height:", String.valueOf(img.getIntrinsicHeight())); float aspect = ((float)img.getIntrinsicWidth()) / ((fl

## Machine accuracy estimates

Some people say that machine epsilon for double precision floating point numbers is 2^-53 and other (more commonly) say its 2^-52. I have messed around estimating machine precision using integers besides 1 and aproaching from above and below (in matl

## C ++ gamedev: truncate float to int

I'm making a game in C++ where is a tank that can moves on a stage. The tank has an angle (float, in degrees, and i'm supposing the tank is at 0º when his cannon points to the right), a speed (float), and there is a time constant called "deltaT"

## Density of the floating point number - Magnitude of the number

Is it true that the more the floating point number is big (either positive or negative) the less we have bits to encode the decimal digits? Can we encode more decimal digits between 21 and 22 than between 216 and 232? Is there the same count of value

## Representation of floating Python numbers

Possible Duplicate: Floating Point Limitations Hi there, maybe this has a very simple answer. I was playing around with the Python interpreter. >>> 1 1 >>> 1.1 1.1000000000000001 >>> 1.2 1.2 Why 1.1 was interpreted as 1.10000000

## NaN Literal in C?

How do you write an NaN floating-point literal in C?In C99's <math.h> 7.12 Mathematics <math.h> [#5] The macro NAN is defined if and only if the implementation supports quiet NaNs for the float type. It expands to a constant expression of type

## Faster way to find the double multiple

If have the following C function, used to determine if one number is a multiple of another to an arbirary tolerance #include <math.h> #define TOLERANCE 0.0001 int IsMultipleOf(double x,double mod) { return(fabs(fmod(x, mod)) < TOLERANCE); } It wo

## Enter a float value or use the functions of math.h floor *?

I am coding up an implementation of Interpolation Search in C. The question is actually rather simple, I need to use the floating operations to do linear interpolation to find the correct index which will eventually be an integer result. In particula

## Float Variable format

I need to format float value and I only need 2 numbers after point and should be rounded value float first = 7, Second = 3,result; result = first / Second; // result contain 2.33333325 since I need like 2.33 Thanks ?5/3 1.6666666666666667 ?String.For

## floating point accuracy

I have a program written in C# and some parts are writing in native C/C++. I use doubles to calculate some values and sometimes the result is wrong because of too small precision. After some investigation i figured out that someone is setting the flo

## Print increments of 0.1 in C #

Im currently reading Code Complete by Steve McConnell, specifically page 295 on floating-point numbers. When i ran the following code: double nominal = 1.0; double sum = 0.0; for (int i = 0; i < 10; i++) { sum += 0.1; Console.WriteLine("sum: "