[BFCL] Adding actionGemma model handler #610
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HuanzhiMao
reviewed
Aug 27, 2024
added 2 commits
August 29, 2024 13:08
|
Thank you for the PR and welcome! We’re currently busy with a new dataset release. I’ll review your submission later this week and aim to provide feedback by next Monday. Apologies for the delay. |
HuanzhiMao
reviewed
Sep 6, 2024
| """ | ||
|
|
||
| class ActionGemmaHandler(OSSHandler): | ||
| def __init__(self, model_name, temperature=0.001, top_p=1, max_tokens=512, dtype="bfloat16") -> None: |
There was a problem hiding this comment.
According to the huggingface model card, the model is supposed to be run with float32 instead of bflaot16.
HuanzhiMao
reviewed
Sep 7, 2024
Comment on lines
+63
to
+68
| content = f"\n{TASK_INSTRUCTION}\n<end_of_turn>\n" | ||
| # content += f"{FORMAT_INSTRUCTION}\n<end_of_turn>\n\n" | ||
| content += "<unused0>\n" + json.dumps(tools) + "\n<unused1>\n\n" | ||
|
|
||
| content += f"<start_of_turn>user\n{query}<end_of_turn>\n\n" | ||
| return SYSTEM_PROMPT + f"\n{content}\n<start_of_turn>assistant" |
There was a problem hiding this comment.
From the model card:
sample response from applied chat template
<bos> <start_of_turn>system You are an expert in composing functions. You are given a question and a set of possible functions. Based on the question, you will need to make one or more function/tool calls to achieve the purpose. If none of the functions can be used, point it out and refuse to answer. If the given question lacks the parameters required by the function, also point it out.<end_of_turn> <start_of_turn>user अमेरिका के राष्ट्रपति कौन है?<end_of_turn> <unused0> [{"name": "get_weather", "description": "Get the current weather for a location", "parameters": {"location": {"type": "string", "description": "The city and state, e.g. San Francisco, New York"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The unit of temperature to return"}}}, {"name": "search", "description": "Search for information on the internet", "parameters": {"query": {"type": > "string", "description": "The search query, e.g. 'latest news on AI'"}}}]<unused1> <start_of_turn>assistant
This seems different from the one formatted here, especially the SYSTEM_PROMPT.
HuanzhiMao
requested changes
Sep 7, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ActionGemma is a Gemma2 9b based fine-tuned model for function calling. This PR adds model handler for the it.