Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare libraries when several sequences are generated #19

Open
rlouf opened this issue Aug 14, 2024 · 0 comments
Open

Compare libraries when several sequences are generated #19

rlouf opened this issue Aug 14, 2024 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@rlouf
Copy link
Member

rlouf commented Aug 14, 2024

We currently only compare libraries for a single generation. However, the main advantage of Outlines is that generation time becomes negligible once the index has been compiled. We should also compare timings for 100 generations for each pattern.

To make sure that the conditions are as close as possible to actually generating text we should generate fake entries using the faker library. Failure to do so would overlook lm-format-enforcer's main downside.

@rlouf rlouf added enhancement New feature or request help wanted Extra attention is needed labels Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant