Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MIPROv2(v2) -- Big Updates to the MIPROv2 Optimizer #1522

Merged
merged 25 commits into from
Sep 22, 2024
Merged

MIPROv2(v2) -- Big Updates to the MIPROv2 Optimizer #1522

merged 25 commits into from
Sep 22, 2024

Conversation

XenonMolecule
Copy link
Collaborator

This PR makes the following updates to MIPROv2:

  • Better delineation of train/validation/test set. The train set is used for bootstrapping demonstrations and your data summary. The validation set is used for the actual optimization. The returned program will be the one that performs best on your validation set. This aligns with how other optimizers use train/val/test splits in DSPy. Note: this means you should typically use a larger validation than a training set.
  • Improved Program Awareness. Removed all dependence on Regex for program awareness. Instead, your entire module code is shown to the LM, which is asked to summarize the particular component (rather than using regex to select the part of the code to view). More checks were added to program awareness to catch exceptions and continue rather than breaking the optimization.
  • Improved Random Prompt Tip Selection. Every prompt generated can use a new prompting tip rather than one tip for the whole run of MIPROv2.
  • Better input validation. You can't run MIPROv2 with fewer than 2 data points, request more evaluations than iterations, etc.
  • Quality of life improvements. If the prompt/task models are not specified, default to the default model. Re-ordering the parameters used in init() and compile() to make it more intuitive and consistent with bootstrap fewshot. Better and more informative logs.
  • HackerCup Demo Added HackerCup optimization with MIPROv2 demo
  • MIPROv2 Docs Improved documentation for MIPROv2

Now, the only remaining question is what to call it. MIPROv2v2? MIPROv2.1? Perhaps just MIPROv2 is easiest

@okhat okhat merged commit c1729b2 into main Sep 22, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants