`acos` domain is in range [-1.0, 1.0]. Due to inaccuracies the value `angleCos` may be slightly outside that range for a correct matrix and `acos(angleCos)` produces `NaN` in that case. The fix is we check `angleCos` value and return `acos(1)` for `angleCos > 1` and `acos(-1)` for `angleCos < -1`. The original code checked only for `angleCos` close to `1.0` and returned an incorrect value for `acos(1)`, which is `0`, not `pi/4`.master
parent
6347f62261
commit
acab24129d
1 changed files with 10 additions and 2 deletions
Loading…
Reference in New Issue