Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rand via scaled random integers #82

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

rand via scaled random integers #82

wants to merge 3 commits into from

Conversation

milankl
Copy link
Member

@milankl milankl commented Sep 19, 2024

fixes #79

Question remains @oscardssmith: Why write (1)

BFloat16(Float32(rand(rng, UInt8)) * Float32(0x1.0p-8))

and not just (2)

rand(rng, UInt8) * BFloat16(0x1.0p-8)

?
From a precision perspective they are the same (no rounding errors in BFloat16 arithmetic because you exactly hit eps/2, eps, 3eps/2, n*eps/2 which are all perfectly representable) but in (1) the actual multiplication is hardcoded in Float32
which avoids the conversion from BFloat16(0x1.0p-8) to Float32 if BFloat16 is not natively supported, I believe? Meaning it's probably faster if bfloat16 is not natively supported. Whereas with (2) we would have the advantage that LLVM's bfloat type would decide what to do depending on the hardware meaning it would be faster on devices with bfloat16 support (as multiplication is then not done with Float32?)

@oscardssmith
Copy link
Member

2 is better

Copy link

codecov bot commented Sep 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.84%. Comparing base (ee51ce2) to head (9ddc0a5).

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #82      +/-   ##
==========================================
+ Coverage   66.66%   66.84%   +0.18%     
==========================================
  Files           3        3              
  Lines         186      184       -2     
==========================================
- Hits          124      123       -1     
+ Misses         62       61       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve precision of rand sampling without conversion
2 participants