AnotherSam
Posts: 14
Joined: Thu Aug 12, 2021 2:37 pm

Converting decimals to large integers for more precision

Wed Sep 15, 2021 6:54 pm

Hello all,

I have a project where I need to calculate fractions from decimals. On my raspberry pi 4 there is a great Fractions module, but no such luck with mciropython. Regardless it isn't a hard task, but I ran into an issue I can't seem to solve. Below is the code:
To TRY and avoid the complications of floating numbers not being very precise, my plan was to "convert" the floating number to an integer by multiplying by 10 enough times. When the function createFraction tries to use, or print, the argument, decimal, the maximum number of characters it will use is 8. I've scoured the net for a bit with no real success, so I thought I'd try and reach out for a solution. Is there a way to force micropyton to use more than 8 characters without have to specific number? Thanks in advance

Code: Select all

class fraction:
    def __init__(self):
        self.WHOLE = 1
        self.NUM = 1
        self.DEM = 1
def createFraction(frac,decimal):
    frac.WHOLE = int(decimal)
    count = 0
    print("decimal: ", decimal)
    while decimal*pow(10,count) % 1 != 0:
        print("loop# :",count," temp: ",decimal*pow(10,count))
        count = count + 1
    testint = int(decimal*pow(10,count))
    tempint = int (frac.WHOLE*pow(10,count))
    print(testint-tempint)

    
sam = fraction()
createFraction(sam,57.1234567)        
 
Below is the output from the code

Code: Select all

decimal:  57.12346
loop# : 0  temp:  57.12346
loop# : 1  temp:  571.2346
loop# : 2  temp:  5712.346
loop# : 3  temp:  57123.45
loop# : 4  temp:  571234.6
loop# : 5  temp:  5712346.0
123456

dbrion06
Posts: 682
Joined: Tue May 28, 2019 11:57 am

Re: Converting decimals to large integers for more precision

Wed Sep 15, 2021 7:13 pm

Floats are not very accurate...(7 decimal digits)

User avatar
scruss
Posts: 4379
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Converting decimals to large integers for more precision

Wed Sep 15, 2021 7:22 pm

Yes, MicroPython uses single-precision floats by default, so the internal representation of your decimal number may be quite different to the numbers you input. When you convert that to a fraction of two integers, it's probably not going to be made up of the integers you expect.

Try your code with a number like 0.2 that can't be represented exactly in binary floating point and see if you get the correct result.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

hippy
Posts: 10754
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Converting decimals to large integers for more precision

Thu Sep 16, 2021 12:48 pm

Building MicroPython to support double precision - viewtopic.php?f=146&t=319525

An alternative is to express float literals as string and have your code convert them to whatever precision you implement -

createFraction(sam,"57.12345678901234567890134567890123456789")

Remove the decimal point and you have a bignum integer. Keep track of where that was and you have an exponent to go with it.

AnotherSam
Posts: 14
Joined: Thu Aug 12, 2021 2:37 pm

Re: Converting decimals to large integers for more precision

Thu Sep 16, 2021 6:55 pm

scruss wrote:Try your code with a number like 0.2
It works just fine for all numbers less than 8 characters long. Anything after 8 gets truncated and that is what i need to fix.
hippy wrote:
Thu Sep 16, 2021 12:48 pm
Remove the decimal point and you have a bignum integer. Keep track of where that was and you have an exponent to go with it.
That was exactly what I was trying to do. However, at some point the float needs to be converted to a string, and I'd rather not do it manually. simply using str(float) truncates the string, and that is the limitation i am trying to solve

User avatar
scruss
Posts: 4379
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Converting decimals to large integers for more precision

Sat Sep 18, 2021 3:23 am

Single precision floating point is only good to 6-8 significant figures, so adding more is adding noise that doesn't belong.

You can create a string with more decimal places using formats. In Python 3:

Code: Select all

>>> s=("%12.10f") % 0.2
>>> s
'0.2000000000'
But this can go sideways once you add more figures:

Code: Select all

>>> s=("%20.18f") % 0.2
>>> s
'0.200000000000000011'
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

ejolson
Posts: 8289
Joined: Tue Mar 18, 2014 11:47 am

Re: Converting decimals to large integers for more precision

Sat Sep 18, 2021 5:54 am

AnotherSam wrote:
Thu Sep 16, 2021 6:55 pm
scruss wrote:Try your code with a number like 0.2
It works just fine for all numbers less than 8 characters long. Anything after 8 gets truncated and that is what i need to fix.
hippy wrote:
Thu Sep 16, 2021 12:48 pm
Remove the decimal point and you have a bignum integer. Keep track of where that was and you have an exponent to go with it.
That was exactly what I was trying to do. However, at some point the float needs to be converted to a string, and I'd rather not do it manually. simply using str(float) truncates the string, and that is the limitation i am trying to solve
If you recompile the Python interpreter as discussed in the thread

viewtopic.php?f=146&t=319525

the floating point numbers will be good to about 15 rather than the current limit of 7 decimal digits. In my opinion, since the performance penalty appears to be less than 5 percent, double precision should actually be the default.

If you want to exactly convert decimals to rational numbers, it's almost required for the decimals to be described by strings. Given such a string, count how many digits appear after the decimal point to obtain a denominator of the form 10^n and then remove the decimal point to obtain the numerator. Then, if necessary, use the Euclidean algorithm to put the fraction in lowest terms.

hippy
Posts: 10754
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Converting decimals to large integers for more precision

Sat Sep 18, 2021 10:34 am

AnotherSam wrote:
Thu Sep 16, 2021 6:55 pm
hippy wrote:
Thu Sep 16, 2021 12:48 pm
Remove the decimal point and you have a bignum integer. Keep track of where that was and you have an exponent to go with it.
That was exactly what I was trying to do. However, at some point the float needs to be converted to a string, and I'd rather not do it manually. simply using str(float) truncates the string, and that is the limitation i am trying to solve
You have two primary options. The easy way; doing it as a user defined class where you will need to implement everything in that class and use those class methods for loading literals, converting to and from integers and strings, will need to use those methods when using your big float numbers and variables.

The harder way is to start hacking under the hood to obtain what you want, making big floats a more integrated MicroPython feature. You can do some tweaking to make 'str()' and other methods handle your big floats automatically, but the best way would be to create a floating point implementation which handles big floats in addition to single and double precision.

That's not going to be simple, will require a lot of work. But let's step back.

The only reason you are even thinking of doing any of this appears to be because you cannot get the Fractions module, which works with Python on a Raspberry Pi, to work with MicroPython on a Pico. It seems to me that if you had a Fractions module things would just work on a Pico and there would be no need to create or use any workarounds

It looks to me like the solution is making the Fractions module work for MicroPython. Solve that and your problem is resolved, and it will also help others who would like to use it.

Is that correct ?

User avatar
scruss
Posts: 4379
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Converting decimals to large integers for more precision

Sat Sep 18, 2021 2:06 pm

There was someone on the MicroPython Forum working on a Fractions module a while back. I don't know if it ever got released.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

hippy
Posts: 10754
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Converting decimals to large integers for more precision

Sat Sep 18, 2021 4:35 pm

I think the Python class for "fractions" is here - https://github.com/python/cpython/blob/ ... actions.py

Seems too much there which MicroPython doesn't support to make it work as is but it may be possible to refactor it to only use what MicroPython provides for and not all of it may need to be implemented to be usable.

I am guessing that's what the OP was trying to do when they ran into their problem.

User avatar
scruss
Posts: 4379
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Converting decimals to large integers for more precision

Sun Sep 19, 2021 2:10 am

ejolson wrote:
Sat Sep 18, 2021 5:54 am
If you want to exactly convert decimals to rational numbers, it's almost required for the decimals to be described by strings. Given such a string, count how many digits appear after the decimal point to obtain a denominator of the form 10^n and then remove the decimal point to obtain the numerator. Then, if necessary, use the Euclidean algorithm to put the fraction in lowest terms.
Something like this, then?

Code: Select all

#!/usr/bin/python3
# -*- coding: utf-8 -*-
# dectorat - decimal string to rational
# scruss, 2021-09

def dectorat(s):
    numerator = 0
    denominator = 1
    sign = 1
    common = 1
    import re
    import math  # see !!WARNING!! below
    s = re.sub("0+$", "", str(s).strip())
    if s.find("-") == 0:
        # process sign
        sign = -1
        s = s[1::]

    if s.find(".") < 0:
        # whole number
        numerator = int(s)
    else:
        # has decimal
        (whole, decimal) = s.split(".")
        denominator = int("1" + "0" * len(decimal))
        numerator = int(decimal) + (int(whole) * int(denominator))
        # !!WARNING!! micropython has no math.gcd() function
        # implementing one that works is an exercise for the reader
        common = math.gcd(numerator, denominator)
    return (sign * numerator//common, denominator//common)


# some test values
for i in [" -12.340000000         ", 23.45,
          "-0.0000000000000000000002370",
          "93", -393, "0.0000000000000000000002500", 0.2]:
    print(i, " => ", dectorat(i))
which gives:

Code: Select all

 -12.340000000           =>  (-617, 50)
23.45  =>  (469, 20)
-0.0000000000000000000002370  =>  (-237, 1000000000000000000000000)
93  =>  (93, 1)
-393  =>  (-393, 1)
0.0000000000000000000002500  =>  (1, 4000000000000000000000)
0.2  =>  (1, 5)
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

lurk101
Posts: 974
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: Converting decimals to large integers for more precision

Sun Sep 19, 2021 2:52 am

scruss wrote:
Sun Sep 19, 2021 2:10 am

Code: Select all

        # !!WARNING!! micropython has no math.gcd() function
        # implementing one that works is an exercise for the reader
        

Code: Select all

def gcd(a, b):
    while b:
        a, b = b, a%b
    return a

User avatar
scruss
Posts: 4379
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Converting decimals to large integers for more precision

Sun Sep 19, 2021 4:14 pm

lurk101 wrote:
Sun Sep 19, 2021 2:52 am

Code: Select all

def gcd(a, b):
    while b:
        a, b = b, a%b
    return a
Doesn't work so well for negative numbers: for that you'd want return max(-a, a).

I wanted to keep it simple as a single function and was testing it on desktop. I kept to the very limited regex functions that micropython has for cleaning up the input string, tho. There were a lot of gotchas in dealing with negative inputs, too.

If this is useful to someone, great - please use it. Writing modules/objects has never been my deal, for $reasons. It's perhaps too simple in that it only understands a (bigint) numerator and denominator. It's got no concept of storing a mixed fraction like 4⅔, but maybe that's an output problem.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

ejolson
Posts: 8289
Joined: Tue Mar 18, 2014 11:47 am

Re: Converting decimals to large integers for more precision

Sun Sep 19, 2021 4:43 pm

scruss wrote:
Sun Sep 19, 2021 4:14 pm
lurk101 wrote:
Sun Sep 19, 2021 2:52 am

Code: Select all

def gcd(a, b):
    while b:
        a, b = b, a%b
    return a
Doesn't work so well for negative numbers: for that you'd want return max(-a, a).

I wanted to keep it simple as a single function and was testing it on desktop. I kept to the very limited regex functions that micropython has for cleaning up the input string, tho. There were a lot of gotchas in dealing with negative inputs, too.

If this is useful to someone, great - please use it. Writing modules/objects has never been my deal, for $reasons. It's perhaps too simple in that it only understands a (bigint) numerator and denominator. It's got no concept of storing a mixed fraction like 4⅔, but maybe that's an output problem.
This looks like a great start to me.

Now one needs to teach MicroPython to add, subtract, multiply, divide and compare fractions. Since I'm not a pythonista, I'm wondering whether MicroPython has the type of operator overloading that allows standard algebraic notation with a user-created fraction type.

I wonder what application the person who made the original post had in mind.

hippy
Posts: 10754
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Converting decimals to large integers for more precision

Sun Sep 19, 2021 4:46 pm

This is my quick and dirty implementation of a 'big float' class which keeps maximum accuracy internally. No division because there's only so many hours in a day -

Code: Select all

#!/usr/bin/python3

import sys

class BigFloat():
  def __init__(self, s):
    if isinstance(s, BigFloat):
      num = s.num
      exp = s.exp
    else:
      s = str(s).upper()
      n = s.find("E")
      if n <= 0:
         exp = 0
      else:
         exp = int(s[n+1:])
         s = s[:n]
      n = s.find(".")
      if n < 0:
        num = int(s)
      else:
        num = int(s[:n] + s[n+1:])
        exp = exp - len(s) + n + 1
    self.num = num
    self.exp = exp
    self.reduce()

  def str(self, exponent=False):
    s = str(self.num)
    n = self.exp
    if   exponent     : return s + "E" + str(n)
    elif n >= 0       : return s + ("0" * n) + ".0"
    elif len(s) <= -n : return "0." + ("0" * (-n - len(s))) + s
    else              : return s[:n] + "." + s[n:]
  def int(self):
    s = self.str()
    return int(s[:s.find(".")])
  def float(self):
    return float(self.str(True))

  def reduce(self):
    if self.num != 0:
      while (self.num % 10) == 0:
        self.num //= 10
        self.exp += 1

  def balance(lhs, rhs):
    while lhs.exp > rhs.exp:
       lhs.num *= 10
       lhs.exp -= 1
    while lhs.exp < rhs.exp:
       rhs.num *= 10
       rhs.exp -= 1

  def add(self, s):
    opr = BigFloat(s)
    self.balance(opr)
    self.num += opr.num
    self.reduce()
  def sub(self, s):
    opr = BigFloat(s)
    self.balance(opr)
    self.num -= opr.num
    self.reduce()
  def mpy(self, s):
    opr = BigFloat(s)
    self.num *= opr.num
    self.exp += opr.exp
    self.reduce()

  def eval_rpn(lst):
    stack = []
    for this in lst:
      this = this.upper()
      if this in ["+", "-", "*", "ADD", "SUB", "MPY"]:
        tos = stack.pop()
        nos = stack.pop()
        if   this in ["+", "ADD"] : nos.add(tos)
        elif this in ["-", "SUB"] : nos.sub(tos)
        elif this in ["*", "MPY"] : nos.mpy(tos)
        stack.append(nos)
      else:
        stack.append(BigFloat(this))
    return stack.pop()

if __name__ == "__main__":
  if len(sys.argv) > 1:
    f = BigFloat.eval_rpn(sys.argv[1:])
    print("'{}' '{}' | {} | {}".format(f.str(True), f.str(), f.int(), f.float()))

Code: Select all

pi@Pi3B:~/apps/bigfloat $ ./bigfloat.py 0.999999999999999999
'999999999999999999E-18' '0.999999999999999999' | 0 | 1.0
pi@Pi3B:~/apps/bigfloat $ ./bigfloat.py 0.999999999999999999 1e-17 add
'1000000000000000009E-18' '1.000000000000000009' | 1 | 1.0
pi@Pi3B:~/apps/bigfloat $ ./bigfloat.py 0.999999999999999999 1e-18 add
'1E0' '1.0' | 1 | 1.0
pi@Pi3B:~/apps/bigfloat $ ./bigfloat.py 0.999999999999999999 1e-19 add
'9999999999999999991E-19' '0.9999999999999999991' | 0 | 1.0

User avatar
scruss
Posts: 4379
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Converting decimals to large integers for more precision

Sun Sep 19, 2021 9:05 pm

ejolson wrote:
Sun Sep 19, 2021 4:43 pm
I wonder what application the person who made the original post had in mind.
I can't second-guess that!

There was a sorta-related post in the MicroPython forum a couple of months back where someone wanted trailing zeroes retained so that measured precision was retained. I'd almost forgotten about that convention and hadn't used it since grammar school.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

Return to “General”