"warning":"This model version is deprecated. Migrate before January 4, 2024

[OldManMeta]

Describe the bug

Im using v3.3.0 attempting to make a call with text-davinci-003

 const result = await openai.createCompletion({
            model: "text-davinci-003",
            prompt: ` ${promptValue} ${prompt}`,
            max_tokens: 2048,
            temperature: 0,
            top_p: 1,
            n: 1,
            stream: true,
        }, { responseType: 'stream' });

I get this sort of return: {"warning":"This model version is deprecated. Migrate before January 4, 2024 to avoid disruption of service. Learn more https://platform.openai.com/docs/deprecations"......

I go to my account and try then use - gpt-3.5-turbo-instruct for the model, and promptly get a response of Model does not exist.

The information provided in the blog posts, says this is a drop in replacement. I'm kinda stuck, and I don't believe an API call should return a warning as the object and a 200 response.

To Reproduce

1. Create a Node application
2. Add the code to call the API
3. Make the call and watch the result

Code snippets

try {
        const result = await openai.createCompletion({
            model: "text-davinci-003",
            prompt: ` ${promptValue} ${prompt}`,
            max_tokens: 2048,
            temperature: 0,
            top_p: 1,
            n: 1,
            stream: true,
        }, { responseType: 'stream' });
        
        result.data.on('data', data => {
            console.log(data);
            const lines = data.toString().split('\n').filter(line => line.trim() !== '');
            for (const line of lines) {
                const message = line.replace(/^data: /, '');
                if (message === '[DONE]') {
                    res.end();
                    return; // Stream finished
                }
                try {
                    const parsed = JSON.parse(message);
                    console.log(parsed.choices[0].text);
                    let responseData = parsed.choices[0].text;
                    res.write(`data: ${responseData}\n\n`);
                } catch(error) {
                    console.error('Could not JSON parse stream message', message, error);
                }
            }
        });



        
    } catch (error) {
        if (error.response?.status) {
            console.error(error.response.status, error.message);
            error.response.data.on('data', data => {
                const message = data.toString();
                try {
                    const parsed = JSON.parse(message);
                    console.error('An error occurred during OpenAI request: ', parsed);
                } catch(error) {
                    console.error('An error occurred during OpenAI request: ', message);
                }
            });
        } else {
            console.error('An error occurred during OpenAI request', error);
        }
    }

### OS

Windows 11

### Node version

Node 18.12.1

### Library version

v3.3.0

[and-zverev]

The same here

This is definitely a bug.
Except for warning, which is completely useless and not needed in API responses, the response chunks themselves come "broken". And this is the main problem.
Examples of parsed chunks in the form of lines below:

[
  'data: {"warning":"This model version is deprecated. Migrate before January 4, 2024 to avoid disruption of service. Learn more https://platform.openai.com/docs/deprecations","id":"cmpl-7j1ybqv0XdxqExtow8VN2E3QQmGho","object":"text_complet'
],
[
  'ion","created":1690966713,"choices":[{"text":"\\n","index":0,"logprobs":null,"finish_reason":null}],"model":"text-davinci-003"}',
  'data: {"warning":"This model version is deprecated. Migrate before January 4, 2024 to avoid disruption of service. Learn more https://platform.openai.com/docs/deprecations","id":"cmpl-7j1ybqv0XdxqExtow8VN2E3QQmGho","object":"text_complet'
],
[
  'ion","created":1690966713,"choices":[{"text":"","index":0,"logprobs":null,"finish_reason":"stop"}],"model":"text-davinci-003"}',
  'data: [DONE]'
]

As you can see, the data comes in two parts: each new chunk contains the end of the previous one and the beginning of the current one. And of course, this can't be parsed with a simple JSON.parse().

Before, chunks came entirely, for example:

[
  'data: {"id":"cmpl-7j1ybqv0XdxqExtow8VN2E3QQmGho","object":"text_completion","created":1690966713,"choices":[{"text":"\\n","index":0,"logprobs":null,"finish_reason":null}],"model":"text-davinci-003"}'
],
[
  'data: [DONE]'
]

Is anyone working on this?

[kevinccbsg]

It seems completion is deprecated, and they recommend using chat completion.

If you change the code to use createChatCompletion and change the model to the one they recommend here (gpt-3.5-turbo-instruct):

you will get an error that the model does not exist. You can check that by calling the https://api.openai.com/v1/models.

What I did was change the createComplation to createChatCompletion.

However, I have to change a little bit the code that processes the response in my project. This is the only workaround that I was able to make it work.

[OldManMeta]

@and-zverev @kevinccbsg thanks for the responses and confirming same - beyond just changing the model, for me see that the TPM is vastly reduced from 150K to 40K. I assume that can be increased on request.

I also note that gpt-3.5-turbo is not the equivalent model of text-davinci-003 as shown here:
https://openai.com/blog/gpt-4-api-general-availability

Changing to createChatCompletion, using the instruct model: (which as you show requires a change to use messages in the fucntion call object)

An error occurred during OpenAI request:  {
  error: {
    message: 'The model `gpt-3.5-turbo-instruct` does not exist',
    type: 'invalid_request_error',
    param: null,
    code: 'model_not_found'
  }
}

Why depreciate something before the declared replacement is broadly available?

The docs at the moment also show that it is still available :
https://platform.openai.com/docs/api-reference/completions/create?lang=node.js

I get things have to change, and that fantastic to see progress, but breaking changes need hard changes to instructions. All of the document examples are still showing the depreciated models - someone needs to think about version details on those pages to remove all doubts on what will work.

From how I now see it, there will be no instruct specific models, just gpt-3.5 turbo variants using chat completions. The cost of these models does appear cheaper than the older davinci ones, but I'm unsure at the moment if the new structure simply uses more tokens on both sides to increase the use and therefore cost.

Cheers

[and-zverev]

Hey @kevinccbsg
You're right about the deprecated API and the migration to the new API. This is all well known.
The point is different - the streaming Completions API is simply no longer working, for the reason which I described above. At the same time, the non-streaming Completions API works as it should.
It seems to me that this problem is related to the core team which deals with the models themselves, since the SDK has not changed. But since I don't know how to contact them, I'm writing here.
It's also strange that none of the SDK maintainers/contributors responded to this ticket.
I can't imagine how people who have the streaming Completions API in production right now feel, since just yesterday everything worked perfectly.
Epic fail
Nothing more to say

[rattrayalex]

This is not an SDK bug, but I am sorry about the frustration caused by this API change.

As the deprecation page mentions:

Note: The recommended replacement, gpt-3.5-turbo-instruct, has not yet launched. Impacted customers will be notified by email when it becomes available.

The streaming chunks being broken that @and-zverev mentions here does look like a possible bug, I'll take that back to the team for discussion.

[OldManMeta]

@rattrayalex the Warning message I suspect is the cause here, as you would be pushing chunks out into the stream at a consistent buffer size, and previously the serialised object would have easily fit the chunk size.

Given that we're supposed to move to createChatCompletion instead, which model is the correct choice to replace text-davinci-003 at the moment? I'd really like to change this out now if possible. Many thanks

[rattrayalex]

For chat completions, gpt-3.5-turbo would be an appropriate replacement.

[schnerd]

Sorry about the trouble here all – the (pre-V4) versions of this SDK never supported streaming natively, so unfortunately we were force to just publish a workaround on github issues:
#18 (comment)

But the workaround is naive and clearly breaks when the buffer size is too large, sorry about that. If you wish to continue using the v3 SDK, this solution might work better for you: #18 (comment)

All that said, I encourage you to check out our v4 beta SDK, which @rattrayalex and team have been working – it has native streaming support and will become the official package version within the next week or two: #182

[rattrayalex]

Just to clarify David's comment a bit, you can use text-davinci-003 with the completions endpoint in the v4 of this SDK like so:

const OpenAI = require("openai");
const openAI = new OpenAI();

async function main() {
  const stream = await openAI.completions.create({
    model: "text-davinci-003",
    prompt: "hello",
    stream: true,
  });
  for await (const part of stream) {
    console.log(part);
  }
}

main();

This streaming implementation does not choke on the warning addition.

If you use the v4, this warning should not break your code, and you have until January to migrate off of text-davinci-003.