Functionary small 2.4 is ignoring the system prompt.

[gunpal5]

Hello,

I was testing the Functionary Small 2.4 model for its capabilities. I found a glitch or bug in the model. I tried to alter the behavior of the model with system prompt. but model is ignoring the instruction in system prompt.

I am using OpenAI client btw.

Thanks,
Gunpal Jain

[gunpal5]

Any comments??

[jeffreymeetkai]

Hi, unfortunately, we have not trained our models with a variety of system prompts. The focus for this model (v2.4) is function-calling and code-generation. Thus, it may not follow instructions in your own system prompts that well.

[gunpal5]

What I wanted to do is use these models for AI Agents reasoning engine. I want to build a smart AI agent using functionary. which will use only to optimize the prompt for the second/Main AI model to write content.

What it needs to do is analyze the prompts and decide whether to call a function or not.

1. if there's no function calling involves it needs to write the original prompt back.
2. if there's a function calling involves, it needs to rewrite the original prompt with the function response. e.g.

<context>
function response
</context>

original prompt

I need to learn how to fine tune it to do the same. I think small model should be fine for this type of scenario.

Edit:
I'll try to hardcode the behavior I want in the block diagram