OTClient OTCv8 Packet size limit game_store

[peteralto]

I'm using a game_store module in OTCv8, however, when creating a category with more than 230 products it apparently exceeds some packet limit (32KB).

ERROR: ProtocolGame parse message exception (51630 bytes, 9304 unread, last opcode is 0x32 (50), prev opcode is 0x32 (50)): InputMessage eof reached
Packet has been saved to packet.log, you can use it to find what was wrong. (Protocol: 1100)

Is there any way to increase the size of this packet?

 

[Gesior.pl]

Is there any way to increase the size of this packet?

You can increase packet size to 65KB with simple server edit:

OTCv8 detection, custom features packet, bigget packet size and bug f… · OTCv8/forgottenserver@2839d4d

…ix for rsa reader

github.com

You can increase packet size to 4 GB with OTCv8 feature GamePacketSizeU32 (and multiple server side changes to send/receive packets with U32 size), but it's a bad idea.
Your OTC store on server side has to generate that XX KB data and convert it to JSON. It takes time. On cheap VPSes big OTCv8 store offer may lag whole OTS for 0.1 sec. OTCv8 store implementation is not designed for big store offer.

RL Tibia store sends given category offers, when you click category in client. OTCv8 store sends all categories and their offers each time you open store in OTC.
RL Tibia store sends data in binary format, OTCv8 sends data in JSON format, which uses much more transfer and it requires conversion from Lua table to JSON, which makes it execute for XX milliseconds.

 

[peteralto]

You can increase packet size to 65KB with simple server edit:

OTCv8 detection, custom features packet, bigget packet size and bug f… · OTCv8/forgottenserver@2839d4d

…ix for rsa reader

github.com

You can increase packet size to 4 GB with OTCv8 feature GamePacketSizeU32 (and multiple server side changes to send/receive packets with U32 size), but it's a bad idea.
Your OTC store on server side has to generate that XX KB data and convert it to JSON. It takes time. On cheap VPSes big OTCv8 store offer may lag whole OTS for 0.1 sec. OTCv8 store implementation is not designed for big store offer.

RL Tibia store sends given category offers, when you click category in client. OTCv8 store sends all categories and their offers each time you open store in OTC.
RL Tibia store sends data in binary format, OTCv8 sends data in JSON format, which uses much more transfer and it requires conversion from Lua table to JSON, which makes it execute for XX milliseconds.

I tried this change, but I can't login to the client. Error 2

Apparently I can't login when including this part in void ProtocolLogin:nRecvFirstMessage(NetworkMessage& msg)

    uint16_t otclientV8 = 0;
    uint16_t otcV8StringLength = msg.get<uint16_t>();
    if(otcV8StringLength == 5 && msg.getString(5) == "OTCv8") {
        otclientV8 = msg.get<uint16_t>(); // 253, 260, 261, ...
    }

 

[Gesior.pl]

I tried this change, but I can't login to the client. Error 2

Ignore all changes from that commit. They change a lot of things.
All you need is to edit (NETWORKMESSAGE_MAXSIZE value is different for different Tibia protocols ex. 8.0, 10.98, so if you use some 'downgrade' server, it may be different value than 24590 to make that downgrade compatible with official Tibia client that cannot handle bigger packets):

static constexpr int32_t NETWORKMESSAGE_MAXSIZE = 24590;

to:

static constexpr int32_t NETWORKMESSAGE_MAXSIZE = 65500;

OTC accepts any packet size for all protocols. Packet may have some extra bytes (checksum, size, sequence number), so it's set to 65500 on server side for safety (35 bytes not used).

 

[peteralto]

Ignore all changes from that commit. They change a lot of things.
All you need is to edit (NETWORKMESSAGE_MAXSIZE value is different for different Tibia protocols ex. 8.0, 10.98, so if you use some 'downgrade' server, it may be different value than 24590 to make that downgrade compatible with official Tibia client that cannot handle bigger packets):

static constexpr int32_t NETWORKMESSAGE_MAXSIZE = 24590;

to:

static constexpr int32_t NETWORKMESSAGE_MAXSIZE = 65500;

OTC accepts any packet size for all protocols. Packet may have some extra bytes (checksum, size, sequence number), so it's set to 65500 on server side for safety (35 bytes not used).

In reality I always used 65500. Maybe it's not enough.

Would this compression system help in this case?

Add packet compression by nekiro · Pull Request #4870 · otland/forgottenserver

Pull Request Prelude I have followed proper The Forgotten Server code styling. I have read and understood the contribution guidelines before making this PR. I am aware that this PR may be clo...

github.com

 

[Gesior.pl]

Would this compression system help in this case?

Yes. It would compress JSON store offer 3-10 times, so 65KB packet would fit probably ~500 KB of JSON data.
It looks like Tibia 12+ feature, not OTCv8 feature, so it probably won't work in OTCv8.

OTCv8 packets compression - for any tibia protocol - is:

Added packet compresion, fixed visual studio compilation errors · OTCv8/forgottenserver@0ad4a30

A free and open-source MMORPG server emulator written in C++, compatible with OTClientV8 - Added packet compresion, fixed visual studio compilation errors · OTCv8/forgottenserver@0ad4a30

github.com

but it's update to code from previous commit I've posted (so you have to implement it first and make it work):

OTCv8 detection, custom features packet, bigget packet size and bug f… · OTCv8/forgottenserver@2839d4d

…ix for rsa reader

github.com

but as I said, if you add it and make Game Store offer bigger, it will lag whole OTS. Every time someone opens Store in client, it will lag ALL players walking/fighting on OTS.

 

[peteralto]

Yes. It would compress JSON store offer 3-10 times, so 65KB packet would fit probably ~500 KB of JSON data.
It looks like Tibia 12+ feature, not OTCv8 feature, so it probably won't work in OTCv8.

OTCv8 packets compression - for any tibia protocol - is:

Added packet compresion, fixed visual studio compilation errors · OTCv8/forgottenserver@0ad4a30

A free and open-source MMORPG server emulator written in C++, compatible with OTClientV8 - Added packet compresion, fixed visual studio compilation errors · OTCv8/forgottenserver@0ad4a30

github.com

but it's update to code from previous commit I've posted (so you have to implement it first and make it work):

OTCv8 detection, custom features packet, bigget packet size and bug f… · OTCv8/forgottenserver@2839d4d

…ix for rsa reader

github.com

but as I said, if you add it and make Game Store offer bigger, it will lag whole OTS. Every time someone opens Store in client, it will lag ALL players walking/fighting on OTS.

Do you have any idea why I can't log into the client when I add this part in void ProtocolLogin:nRecvFirstMessage(NetworkMessage& msg) ?

    uint16_t otclientV8 = 0;
    uint16_t otcV8StringLength = msg.get<uint16_t>();
    if(otcV8StringLength == 5 && msg.getString(5) == "OTCv8") {
        otclientV8 = msg.get<uint16_t>(); // 253, 260, 261, ...
    }

 

[Mateus Robeerto]

Do you have any idea why I can't log into the client when I add this part in void ProtocolLogin:nRecvFirstMessage(NetworkMessage& msg) ?

    uint16_t otclientV8 = 0;
    uint16_t otcV8StringLength = msg.get<uint16_t>();
    if(otcV8StringLength == 5 && msg.getString(5) == "OTCv8") {
        otclientV8 = msg.get<uint16_t>(); // 253, 260, 261, ...
    }

The part you added is in the wrong place. You should move it down to the correct spot I showed you in the link:
TFS-1.5-Downgrades/src/protocollogin.cpp at 8.60 · nekiro/TFS-1.5-Downgrades (https://github.com/nekiro/TFS-1.5-Downgrades/blob/8.60/src/protocollogin.cpp#L167)
Just add it above the highlighted line I showed (the one in yellow), then recompile. It should work.

 

[peteralto]

This way I can login, but I can no longer use ExtendedOpcode. I think it has to do with this change in void ProtocolGame::login:

    // OTCv8 features and extended opcodes
    if (otclientV8 || operatingSystem >= CLIENTOS_OTCLIENT_LINUX) {
        if(otclientV8)
            sendFeatures();
        NetworkMessage opcodeMessage;
        opcodeMessage.addByte(0x32);
        opcodeMessage.addByte(0x00);
        opcodeMessage.add<uint16_t>(0x00);
        writeToOutputBuffer(opcodeMessage);
    }