Skip to content

Issues: symflower/eval-dev-quality

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

42 Open 79 Closed
42 Open 79 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Enable Ruby tests in Windows CI bug Something isn't working CI GitHub Actions
#334 opened Aug 30, 2024 by bauersimon v0.7.0
Add new subcommand verify that just does repository verification and run that in CI bug Something isn't working
#330 opened Aug 27, 2024 by bauersimon
Kubernetes job watching does not work when erroring bug Something isn't working
#329 opened Aug 27, 2024 by Munsio v0.7.0
@Munsio
Move Dependency installation of Docker into multistage builds CI GitHub Actions enhancement New feature or request
#319 opened Aug 8, 2024 by Munsio
3 tasks
v0.7.0
@Munsio
Copy of evaluation data for kubernetes is not working bug Something isn't working
#312 opened Aug 5, 2024 by Munsio v0.7.0
@Munsio
Follow-up: Use a JSON configuration file to set up an evaluation run enhancement New feature or request
#307 opened Aug 1, 2024 by ruiAzevedo19
1 task
v0.7.0
@ruiAzevedo19
Rethink retry logic for LLM Providers bug Something isn't working
#305 opened Aug 1, 2024 by Munsio v0.7.0
@bauersimon
Reporting tool does not work with /**/ directory globing bug Something isn't working
#304 opened Aug 1, 2024 by Munsio v0.7.0
@ruiAzevedo19
5
Roadmap for v0.7.0 roadmap Collection of issues for a release
#301 opened Jul 31, 2024 by bauersimon 100+ v0.7.0
@zimmski
Ruby support enhancement New feature or request
#300 opened Jul 31, 2024 by ahumenberger
22 of 23 tasks
v0.7.0
@ruiAzevedo19
3
Data visualization based on evaluation CSV files enhancement New feature or request
#296 opened Jul 30, 2024 by ruiAzevedo19
4 of 7 tasks
v0.7.0
@ruiAzevedo19
1
Openrouter Provider preferences bug Something isn't working enhancement New feature or request
#286 opened Jul 25, 2024 by Munsio v0.7.0
@Munsio
2
"symflower unit-tests" timeout error differs between Linux and Windows bug Something isn't working
#280 opened Jul 18, 2024 by ruiAzevedo19
2 tasks
v0.7.0
Follow-up "Check if the testdata repository is valid before running the evaluation, so it is checked just once" refactor Make code cleaner/better
#266 opened Jul 15, 2024 by ruiAzevedo19 v0.7.0
@ruiAzevedo19
Assess failing tests enhancement New feature or request
#235 opened Jul 2, 2024 by zimmski v0.7.0
1
Weight "executed code" more prominently enhancement New feature or request
#233 opened Jul 2, 2024 by zimmski v0.7.0
@bauersimon
4
Multiple model parameter with same value result in multiple evaluations bug Something isn't working question Further information is requested
#220 opened Jun 26, 2024 by Munsio v0.7.0
@Munsio
Evaluation run for all "good open weight models" with all available quantizations and different GPUs enhancement New feature or request
#209 opened Jun 22, 2024 by zimmski v0.7.0
Interactive result comparison enhancement New feature or request
#208 opened Jun 21, 2024 by bauersimon
9 tasks
v0.7.0
Keep individual coverage files and LLM query/responses enhancement New feature or request
#204 opened Jun 19, 2024 by zimmski v0.7.0
@ahumenberger
3
Roadmap for v0.6.0 roadmap Collection of issues for a release
#195 opened Jun 17, 2024 by zimmski 100+ v0.6.0
@zimmski
Evaluation task: TDD enhancement New feature or request
#194 opened Jun 17, 2024 by ahumenberger v0.7.0
Coverage for Java is tracked for lines, while Go is tracked for ranges bug Something isn't working
#193 opened Jun 14, 2024 by bauersimon v0.7.0
4
Deal with dependencies requested by LLMs bug Something isn't working
#174 opened Jun 10, 2024 by ahumenberger v0.7.0
2
ollama_llama_server and other background processes we start must be killed on CTRL+C bug Something isn't working
#164 opened Jun 6, 2024 by zimmski v0.7.0
3
ProTip! Follow long discussions with comments:>50.