Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Socket disconnects on big payload. #5140

Open
faisal-anwar825 opened this issue Jul 18, 2024 · 10 comments
Open

Socket disconnects on big payload. #5140

faisal-anwar825 opened this issue Jul 18, 2024 · 10 comments
Labels
question Further information is requested

Comments

@faisal-anwar825
Copy link

faisal-anwar825 commented Jul 18, 2024

I am using socket.io to create a realtime chat which includes attachments as well. I am sending files in Base64 form. but the issue is that if file size is around 500 to 600 KB. it gets send successfully but when file size increases more than that the sockets disconnects as soon as i send the message, with the error

Transport close: The connection was closed

i have tried increasing the maxHttpBufferSize to 1MB and 100MB but still files > 7, 800 KBs wont upload.

Socket.IO server version: 4.7.2

This is the code from my server

`

    import {
      WebSocketGateway,
      WebSocketServer,
      SubscribeMessage,
      OnGatewayConnection,
      OnGatewayDisconnect,
    } from '@nestjs/websockets';
    import { Server, Socket } from 'socket.io';
    import { SocketsService } from '../sockets.service';
    import { WsGuard } from '@app/common';
    import { ChatDto } from '../dto/chat.dto';
    import { Injectable, UseGuards } from '@nestjs/common';
    
    @Injectable()
    @WebSocketGateway({ cors: '*:*' })
    export class ChatGateway implements OnGatewayConnection, OnGatewayDisconnect {
      @WebSocketServer() public server: Server;
      users = 0;
    
      constructor(
        private readonly socketService: SocketsService,
      ) { }
    
      async onModuleInit() {
        // Call the configureWebSocketServer method when the module has been initialized
        this.configureWebSocketServer();
      }
    
      async handleConnection(client: Socket) {
        // A client has connected
        this.users++;
    
        client.removeAllListeners('disconnect');
    
        client.on('disconnect', () => this.handleDisconnect(client));
        // Notify connected clients of current users
        this.server.emit('users', this.users);
      }
    
      afterInit(server: Server) {
        this.socketService.socket = server;
      }
    
      async handleDisconnect(client: Socket) {
        // A client has disconnected
        this.users--;
        client.removeAllListeners('disconnect');
    
        client.on('disconnect', () => this.handleDisconnect(client));
        // Notify connected clients of current users
        this.server.emit('users', this.users);
      }
    
      @SubscribeMessage('chat')
      @UseGuards(WsGuard)
      async onChat(client: Socket, payload: ChatDto) {
        // expect to receive the file here
      }
    
      async configureWebSocketServer() {
    
        this.server.engine.opts.maxHttpBufferSize = 4e8;
      }
    }

`

@faisal-anwar825 faisal-anwar825 added the to triage Waiting to be triaged by a member of the team label Jul 18, 2024
@Malikrehman00107
Copy link

@rauchg

please !

@Malikrehman00107
Copy link

@dantehemerson

@darrachequesne
Copy link
Member

Hi! I see you are using NestJS, did you increase the body parser size limit?

Reference: https://docs.nestjs.com/faq/raw-body#body-parser-size-limit

@darrachequesne darrachequesne added question Further information is requested and removed to triage Waiting to be triaged by a member of the team labels Jul 19, 2024
@faisal-anwar825
Copy link
Author

faisal-anwar825 commented Jul 19, 2024

Yes i have set bodyparser limit as well. this is my main.ts file

`

  import { NestFactory } from '@nestjs/core';
  import { SocketsModule } from './sockets.module';
  import * as cors from 'cors';
  import * as bodyParser from 'body-parser';
  
  async function bootstrap() {
    const corsOptions = {
      origin: [
        // cors option
      ],
    };
    const app = await NestFactory.create(SocketsModule, { cors: true });
    app.enableCors(corsOptions);
    app.use(cors());
    app.use(bodyParser.json({ limit: '50mb' }));
    app.use(bodyParser.raw({ limit: '50mb' }));
  
  
      await app.listen(3000);
  }
  bootstrap();

`

@Malikrehman00107
Copy link

@darrachequesne

@llaakso
Copy link

llaakso commented Jul 21, 2024

Large packets will cause timeout so you need to split the data to multiple packets. This approach is also good for not to block event loop which often happens with large data. I believe this should be addressed in socket.io level as devs are still wondering this issue. If nothing else, just console warning when the packet is likely too big.

@faisal-anwar825
Copy link
Author

That's understandable for maybe files that are huge in size but a 700KB or 800KB shouldn't need breaking down. no?

@Malikrehman00107
Copy link

@llaakso are you ref like this?

Screenshot 2024-07-22 at 1 09 28 AM

@darrachequesne
Copy link
Member

@faisal-anwar825 @Malikrehman00107 unfortunately I was not able to reproduce the issue: https://github.com/socketio/socket.io/tree/main/examples/nestjs-example

src/views/index.hbs

socket.on("connect", () => {
  // [...]
  socket.emit("big file", "a".repeat(1e7)); // 10 MB
});

src/events/events.gateway.ts

@WebSocketGateway({
  maxHttpBufferSize: 1e8
})
export class EventsGateway {
  // [...]
  @SubscribeMessage('big file')
  handleBigFile(@MessageBody() data: string): void {
    console.log("received", data.length);
  }
}

Same with the onModuleInit hook:

@WebSocketGateway({})
export class EventsGateway {
  // [...]
  async onModuleInit() {
    this.server.engine.opts.maxHttpBufferSize = 1e8;
  }

  @SubscribeMessage('big file')
  handleBigFile(@MessageBody() data: string): void {
    console.log("received", data.length);
  }
}

I believe this should be addressed in socket.io level as devs are still wondering this issue.

@llaakso I guess we could indeed split big payloads into several chunks. Maybe as a custom parser? What do you think?

Reference: https://socket.io/docs/v4/custom-parser/

@Malikrehman00107
Copy link

@darrachequesne we are still facing the issue so I belive the idea from @llaakso is a good one!

we are going to try and will surely update in case it works.

best

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants