Fix shader-interpreter non-IEEE outputs (#459)

* Re-enable non-IEEE shader test

* Fix shader-interpreter RCP/RSQ output

Handle the `-0.0` special-case

* Fix shader-interpreter MIN/MAX output

Takes advantage of min/max's properties regarding non-finites to return
NaN depending on its input position:
```
max(NaN, 2.f) -> NaN
max(2.f, NaN) -> 2
min(NaN, 2.f) -> NaN
min(2.f, NaN) -> 2
```

* Fix shader-interpreter FLR indexing bug

`3 - 1` should be `3 - i`
This commit is contained in:
Wunk 2024-03-11 23:34:02 -07:00 committed by GitHub
parent fe9939689d
commit c89fe05b8a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 24 additions and 8 deletions

View file

@ -113,7 +113,7 @@ SHADER_TEST_CASE("RCP", "[shader][vertex]") {
{nihstro::OpCode::Id::END},
});
// REQUIRE(shader->RunScalar({-0.0f}) == INFINITY); // Violates IEEE
REQUIRE(shader->runScalar({-0.0f}) == INFINITY);
REQUIRE(shader->runScalar({0.0f}) == INFINITY);
REQUIRE(shader->runScalar({INFINITY}) == 0.0f);
REQUIRE(std::isnan(shader->runScalar({NAN})));