Steps to test out V3

[stinoga]

Question

Are there steps documented anywhere for running V3 in your app? I'm trying to test it out via these steps:

1. Pointing to the alpha in my package.json: "@huggingface/transformers": "^3.0.0-alpha.10",
2. npm i
3. cd node_modules/@hugginface/transformers && npm i
4. copy the webpack.config.js from the repo into the node_modules/@hugginface/transformers dir.
5. npm run build in node_modules/@hugginface/transformers dir.

I then run my app, and get the following error:

ERROR in ../../node_modules/@huggingface/transformers/dist/transformers.js 42256:34-64
Module not found: Error: Can't resolve './' in '/node_modules/@huggingface/transformers/dist'
webpack compiled with 1 error

Thanks, I'm excited to test out the latest and greatest!

[nnennajohn]

+1 on this error. Also in 3.0.0-alpha.13

[xenova]

Hi there! 👋 What environment are you running in (OS, build tool, browser, node)? 👀 I haven't seen an error like this before 🤔 Can you share your webpack config?

[stinoga]

Hi there! 👋 What environment are you running in (OS, build tool, browser, node)? 👀 I haven't seen an error like this before 🤔 Can you share your webpack config?

Sure can!

OS: Sonoma 14.6.1
Build tool: Lerna
Browser: Chrome
Node: v22.3.0

Regarding webpack config, we aren't using webpack. The config I coped into transformers dir was from the main branch.

[legut2]

Webpack Error:

> app@1.0.0 build
> webpack

assets by status 676 KiB [cached] 1 asset
runtime modules 318 bytes 2 modules
orphan modules 1.66 MiB [orphan] 1 module
./static/images/js/worker.js + 1 modules 1.68 MiB [built] [code generated]

ERROR in ./node_modules/@huggingface/transformers/dist/transformers.js 32909:34-64
Module not found: Error: Can't resolve './' in '/app/node_modules/@huggingface/transformers/dist'
resolve './' in '/app/node_modules/@huggingface/transformers/dist'
  Parsed request is a directory
  using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist)
    using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist)
      as directory
        existing directory /app/node_modules/@huggingface/transformers/dist
          using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist)
            using path: /app/node_modules/@huggingface/transformers/dist/index
              using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist/index)
                no extension
                  /app/node_modules/@huggingface/transformers/dist/index doesn't exist
                .js
                  /app/node_modules/@huggingface/transformers/dist/index.js doesn't exist
 @ ./static/images/js/worker.js 6:0-53

webpack 5.93.0 compiled with 1 error in 2902 ms

I ran into this issue when using webpack on worker.js into worker.bundle.js. @xenova/transformers on 2.17.2 worked just fine. I tried upgrading to @huggingface/transformers on 3.0.0-alpha.14.

I'm on:
macOS: 14.3.1 (23D60)
Node: v22.7.0
Browser: Firefox

With the following import statement, it works well

import { pipeline } from '@xenova/transformers';

alongside this webpack.config.js file:

const path = require('path');

module.exports = {
  entry: './static/images/js/worker.js',
  output: {
    filename: 'worker.bundle.js',
    path: path.resolve(__dirname, 'static/images/js'),
  },
  mode: 'production',
  target: 'webworker',
  module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader',
          options: {
            presets: ['@babel/preset-env'],
          },
        },
      },
    ],
  },
  resolve: {
    extensions: ['.js'],
  },
};

However, it doesn't work with:

import { pipeline, AutoProcessor, Swin2SRForImageSuperResolution, RawImage } from '@huggingface/transformers';

[sebastien-lempens]

I ran into the same issue, but I’ve figured out the solution!
Just add the following to your Webpack configuration file (or Next.js configuration in my case):

webpack(config, { isServer }) {
	config.resolve.alias['@huggingface/transformers'] = path.resolve(__dirname, 'node_modules/@huggingface/transformers');
        .... 
	return config;
	}

It works like a charm now :)

[do-me]

The error happens only with version 3.0.0-alpha.10 or newer. 3.0.0-alpha.9 works fine for me. Was testing with webpack and in this env:

OS: Sonoma 14.6.1
Browser: Chrome
Node: v22.3.0

[GaLambert]

+1, @sebastien-lempens thanks for the solution!

[stinoga]

The error happens only with version 3.0.0-alpha.10 or newer. 3.0.0-alpha.9 works fine for me. Was testing with webpack and in this env:

OS: Sonoma 14.6.1 Browser: Chrome Node: v22.3.0

This was the temporary solve for me. Since we are using Lerna with CRA, the webpack alias didn't fix things.

Not sure what happened between versions alpha 9 and 10, but that's where this issue starts showing up.