gradients: fix saturation/luma calculations for 32-bit systems (#1470)

* gradients: fix saturation calculation for 32-bit systems

this multiplication has two 512 scale factors as well as 360 and a color difference of up to 255. as such, it can overflow 32 bits, which results in an incorrect saturation value being computed

cast to uint64_t so we have enough bits to multiply without overflow

* gradients: fix luma calculation for 32-bit systems

this multiplication has 0-255 colors at 512 scale factor being multiplied by 10000 (summed across components), then multiplied by 360

this multiplication (360 * 10000 * 255 * 512) overflows a long on 32-bit systems, computing the wrong luma value

cast to uint64_t so we have enough bits to multiply without overflow

---------

Co-authored-by: bi4k8 <bi4k8@github>
This commit is contained in:
bi4k8 2023-03-25 12:28:10 +00:00 committed by GitHub
parent a9cecaf991
commit 2fb7d47039
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 2 deletions

View File

@ -156,7 +156,7 @@ void hsv_gradient_factory::convert_from_scaled_rgb(long *const scaled,
auto value = get_value(scaled);
auto minimum = get_minimum(scaled);
auto chroma = value - minimum;
auto saturation = (SCALE360 * chroma) / value;
long saturation = (SCALE360 * (uint64_t)chroma) / value;
target[0] = get_hue(scaled, chroma, value);
target[1] = saturation;
@ -199,7 +199,7 @@ namespace {
// Using Rec.2020 color space
// Y' = 0.2627 x R + 0.6780 x G + 0.0593 x B
long get_luma(long *const rgb) {
return 360L * (2627L * rgb[0] + 6780L * rgb[1] + 593L * rgb[2]) / 10000L;
return 360L * (uint64_t)(2627L * rgb[0] + 6780L * rgb[1] + 593L * rgb[2]) / 10000L;
}
// Using Rec.2020 color space