Maybe I'm getting rusty...
With .Offset(CurRow, 8)
.Value = Int(MyRecord.TempI * 10) / 10
End With
When the TempI element is 24.8 I see it in my cell as 24.7999992370605 (even with the extra rounding I added above)... What is this???
Weird? Cell shows fractions...
-
- BronzeLounger
- Posts: 1258
- Joined: 03 Feb 2010, 19:59
- Location: Terneuzen, the Netherlands
-
- Administrator
- Posts: 78629
- Joined: 16 Jan 2010, 00:14
- Status: Microsoft MVP
- Location: Wageningen, The Netherlands
Re: Weird? Cell shows fractions...
It's a rounding problem. Has Templ been defined as Single? If so, try defining it as Double instead. The greater precision of the Double data type reduces the incidence of rounding errors (but it doesn't eliminate them entirely, computing precision is always finite).
Best wishes,
Hans
Hans
-
- BronzeLounger
- Posts: 1258
- Joined: 03 Feb 2010, 19:59
- Location: Terneuzen, the Netherlands
Re: Weird? Cell shows fractions...
So the INT(x*10)/10 doesn't fix that???
-
- Administrator
- Posts: 78629
- Joined: 16 Jan 2010, 00:14
- Status: Microsoft MVP
- Location: Wageningen, The Netherlands
Re: Weird? Cell shows fractions...
Nope - the Single data type doesn't store the number 24.8 with sufficient accuracy. Single has a precision of about 7 significant digits, while Excel internally uses the Double data type, which has about 15 significant digits. So if you use Single in your VBA code, you're bound to see rounding errors when you store values in cells.
Best wishes,
Hans
Hans