Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Difficulty in importing { zodResponseFormat } from https://deno.land #972

Open
1 task done
samreid opened this issue Aug 7, 2024 · 11 comments
Open
1 task done

Comments

@samreid
Copy link

samreid commented Aug 7, 2024

Confirm this is a feature request for the Node library and not the underlying OpenAI API.

  • This is a feature request for the Node library

Describe the feature or improvement you're requesting

In testing with structured outputs, import { zodResponseFormat } from 'npm:openai/helpers/zod'; works correctly at runtime (with --unstable, but does not provide types), but import { zodResponseFormat } from 'https://deno.land/x/[email protected]/helpers/zod.ts'; yields runtime resolution errors. I do not have enough expertise to know whether I'm doing something wrong or whether there is a problem in the deno.land for the helpers/zod. Please advise.

Additional context

In testing with the structured outputs, this works correctly (using npm: import resolution):

// deno run --allow-all --unstable src/main-structured.ts

import OpenAI from 'npm:openai';
import { zodResponseFormat } from 'npm:openai/helpers/zod';
import { z } from 'npm:zod';

const openai = new OpenAI();

const WriteFile = z.object( {
  op: z.literal( 'writeFile' ),
  writeFilename: z.string(),
  contents: z.string()
} );

const ReadFile = z.object( {
  op: z.literal( 'readFile' ),
  readFilename: z.string()
} );

// Define an anyOf schema using Zod
const FileOperation = z.object( {
  selectedAction: z.union( [ ReadFile, WriteFile ] )
} );

const x = zodResponseFormat( FileOperation, 'mySchema' )
console.log( JSON.stringify( x, null, 2 ) );

// Use the anyOf schema in response_format
const completion = await openai.beta.chat.completions.parse( {
  model: 'gpt-4o-2024-08-06',
  messages: [
    { role: 'system', content: 'Choose the next command to accomplish the goal.' },
    { role: 'user', content: 'Write a haiku' }
  ],
  response_format: zodResponseFormat( FileOperation, 'mySchema' )
} );

const event = completion.choices[ 0 ].message.parsed;
console.log( event );

However, this does not yield type information and must be run with --unstable. The openai-node documentation says we can import from deno like so:

import OpenAI from 'https://deno.land/x/[email protected]/mod.ts';

However, when trying to import all imports from deno.land like so:

// Note --unstable no longer needed
// deno run --allow-all src/main-structured.ts

import OpenAI from 'https://deno.land/x/[email protected]/mod.ts';
import { zodResponseFormat } from 'https://deno.land/x/[email protected]/helpers/zod.ts';
import { z } from 'https://deno.land/x/[email protected]/mod.ts';
// etc...

We obtain this error at runtime:

~/myProject$ deno run --allow-all src/main-structured.ts
error: Remote modules are not allowed to import local modules. Consider using a dynamic import instead.
  Importing: npm:zod
    at https://deno.land/x/[email protected]/_vendor/zod-to-json-schema/parsers/tuple.ts:1:56

I do not have enough expertise to know whether I'm doing something wrong or whether there is a problem in the deno.land for the helpers/zod. Please advise.

@RobertCraigie
Copy link
Collaborator

@samreid when you say the types don't work, do you get an error or is it resolved to any?

@samreid
Copy link
Author

samreid commented Aug 7, 2024

My IDE IntelliJ hoversense reported any.

image

I tried this code at the beginning:

// deno run --allow-all --unstable src/main-structured.ts

import OpenAI from 'npm:openai';
import { zodResponseFormat } from 'npm:openai/helpers/zod';
import { z } from 'npm:zod';

const openai = new OpenAI();
console.log( openai.hello ); // checking for a type error here

and when running deno check --unstable src/main-structured.ts it reported:

Check file:///Users/samreid/projects/alliterate/src/main-structured.ts
error: TS2305 [ERROR]: Module '"deno:///missing_dependency.d.ts"' has no exported member 'zodResponseFormat'.
import { zodResponseFormat } from 'npm:openai/helpers/zod';
         ~~~~~~~~~~~~~~~~~
    at file:///Users/samreid/projects/alliterate/src/main-structured.ts:4:10

TS2305 [ERROR]: Module '"deno:///missing_dependency.d.ts"' has no exported member 'z'.
import { z } from 'npm:zod';
         ^
    at file:///Users/samreid/projects/alliterate/src/main-structured.ts:5:10

Found 2 errors.

so it found some type errors, but did not catch the openai.hello type error.

@RobertCraigie
Copy link
Collaborator

Thanks for the detailed report and sorry this isn't working in deno right now. Will investigate.

@samreid
Copy link
Author

samreid commented Aug 8, 2024

This looks fixed in [email protected], I tested this and the runtime and IDE intellisense type checking and deno check all worked much better:

// deno run --allow-all src/main-structured.ts

import OpenAI from 'https://deno.land/x/[email protected]/mod.ts';
import { z } from 'https://deno.land/x/[email protected]/mod.ts';
import { zodResponseFormat } from 'https://deno.land/x/[email protected]/helpers/zod.ts';

const openai = new OpenAI();
console.log( openai.hello ); // checking for a type error here

const WriteFile = z.object( {
  op: z.literal( 'writeFile' ),
  writeFilename: z.string(),
  contents: z.string()
} );

const ReadFile = z.object( {
  op: z.literal( 'readFile' ),
  readFilename: z.string()
} );

// Define an anyOf schema using Zod
const FileOperation = z.object( {
  selectedAction: z.union( [ ReadFile, WriteFile ] )
} );

const x = zodResponseFormat( FileOperation, 'mySchema' )
console.log( JSON.stringify( x, null, 2 ) );

// Use the anyOf schema in response_format
const completion = await openai.beta.chat.completions.parse( {
  model: 'gpt-4o-2024-08-06',
  messages: [
    { role: 'system', content: 'Choose the next command to accomplish the goal.' },
    { role: 'user', content: 'Write a haiku' }
  ],
  response_format: zodResponseFormat( FileOperation, 'mySchema' )
} );

const event = completion.choices[ 0 ].message.parsed;
console.log( event );

I saw that this line:

const x = zodResponseFormat( FileOperation, 'mySchema' )

triggers an error:

Deno: Type instantiation is excessively deep and possibly infinite.

But that sounds like a separate issue. Nice work everyone. Closing.

@samreid samreid closed this as completed Aug 8, 2024
@magnetenstad
Copy link

@samreid Did you manage to resolve the other error? I have the same issue.

Type instantiation is excessively deep and possibly infinite.

@samreid
Copy link
Author

samreid commented Aug 12, 2024

I did not investigate it further, but I did commit this type assertion:

        // this any speeds up type checking by about 20 seconds
        response_format: ( zodResponseFormat as any )( totalSchema, 'commandSchema' )

@RobertCraigie RobertCraigie reopened this Aug 14, 2024
@RobertCraigie
Copy link
Collaborator

does this work for you?

type Schema = z.infer<typeof totalSchema>;

@KevinKreps
Copy link

KevinKreps commented Aug 18, 2024

Type instantiation is excessively deep and possibly infinite.deno-ts

Facing this same problem here, where also type checking gets very laggy.

@samreid solution with casting to any works, a nicer solution would be great tho

@RobertCraigie This works independent of that any:

type Schema = z.infer<typeof totalSchema>;

@RobertCraigie
Copy link
Collaborator

@KevinKreps thanks, is any additional information provided in the error message? e.g. a stack trace?

@KevinKreps
Copy link

@RobertCraigie

The best I can think of right now is running deno check my-file.ts --log-level=debugwhich dumps a lot of text ending with this. Hit me up if you have a better idea to get more useful info out of it or if you need the complete text dump of all the debug logs.

DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable.tsx")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable/index.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable/index.tsx")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable/index.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@types/typescript__lib-deno/unstable/package.json")
DEBUG TS - host.fileExists("cache:///node_modules/@types/typescript__lib-deno/unstable.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@types/typescript__lib-deno/unstable/index.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable.js")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable.jsx")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable/index.js")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/unstable/index.jsx")
DEBUG TS - host.getSourceFile("asset:///lib.deno.unstable.d.ts", Latest)
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel/package.json")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel.tsx")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel/index.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel/index.tsx")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel/index.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@types/typescript__lib-deno/broadcast_channel/package.json")
DEBUG TS - host.fileExists("cache:///node_modules/@types/typescript__lib-deno/broadcast_channel.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@types/typescript__lib-deno/broadcast_channel/index.d.ts")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel.js")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel.jsx")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel/index.js")
DEBUG TS - host.fileExists("cache:///node_modules/@typescript/lib-deno/broadcast_channel/index.jsx")
DEBUG TS - host.getSourceFile("asset:///lib.deno.broadcast_channel.d.ts", Latest)
DEBUG TS - <<< exec stop
DEBUG RS - deno::tools::check:227 - Compilation statistics:
  Files: 250
  Nodes: 172389
  Identifiers: 59692
  Symbols: 32619
  Types: 85
  Instantiations: 0
  Parse time: 333
  Bind time: 126
  Check time: 0
  Emit time: 0
  Total TS time: 459
  Compile time: 502

error: TS2589 [ERROR]: Type instantiation is excessively deep and possibly infinite.
    const responseFormat = zodResponseFormat(
                           ^
    at ...my-file.ts
    ```

@RobertCraigie
Copy link
Collaborator

thanks @KevinKreps! can you / someone in this thread file an issue with Deno? I don't think we're doing anything too weird here so I suspect there's some bug with the deno typescript implementation.

unfortunately it's unlikely we're going to be able to prioritise investigating this in the short term, so any PR would be really appreciated :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants