• There is NO official Otland's Discord server and NO official Otland's server list. The Otland's Staff does not manage any Discord server or server list. Moderators or administrator of any Discord server or server lists have NO connection to the Otland's Staff. Do not get scammed!

Tibia 11 .dat File Structure

jo3bingham

Excellent OT User
Joined
Mar 3, 2008
Messages
1,103
Solutions
14
Reaction score
772
GitHub
jo3bingham
https://tibiapf.com/showthread.php?101-dat-Structure

Here you can see my progress in understanding the new .dat file structure. My 4th post (Update #3) contains a basically-complete breakdown of the structure. I still have a few flags to find, and some unknown values to figure out, but, for the most part, I have the whole structure decoded.
 
Update #4

Thanks to Daniel (blackd), the weird numbers (like with object ids, and some flags) has been solved.

If a value is greater-than-or-equal to 0x80 then there will be another value following it. If that value is greater-than-or-equal to 0x80 there will be another value following it. So on and so forth until the next value is less-than 0x80, but it should never be more than four values. Furthermore, the value of each byte can be calculated with the following formula: ((128^x)*(n))-(128^x) where x is the index of the current byte (0-based) and n is the value of the byte. Using recursion, and adding each calculated value, we can get the final value.

For example, the object id of rope is 0xBB 0x17 which, when read as a 2-byte integer gives the value 6075, but we know that the item id of rope is 3003. If we plug 0xBB (187) into the equation we get ((128^0)*(187))-(128^0)=186. We use 0 for x because 0xBB is the first byte and we count up from 0. Now, if we plug 0x17 (23) into the equation we get ((128^1)*(23))-(128^1)=2816. We use 1 for x here because it's the second byte but we count up from 0. Take the sum of both values and you get 186+2816=3002. The item id of rope is actually 3003, so we just need to add 1 to the sum.

pseudo-code:
Code:
int readValue(byte[] buffer, int position, int index = 0) {
    byte num1 = buffer[position];
    int num2 = ((128^index)*(num1))-(128^index);

    if (index == 0) {
        num2++;
    }

    if (num1 < 128) {
        return num2;
    }

    return num2 + readValue(buffer, position++, index++);
}

I'm not the best at formulas, so if anyone can write a better one to remove that first if-statement that would be great.
 
Update #4

Thanks to Daniel (blackd), the weird numbers (like with object ids, and some flags) has been solved.

If a value is greater-than-or-equal to 0x80 then there will be another value following it. If that value is greater-than-or-equal to 0x80 there will be another value following it. So on and so forth until the next value is less-than 0x80, but it should never be more than four values. Furthermore, the value of each byte can be calculated with the following formula: ((128^x)*(n))-(128^x) where x is the index of the current byte (0-based) and n is the value of the byte. Using recursion, and adding each calculated value, we can get the final value.

For example, the object id of rope is 0xBB 0x17 which, when read as a 2-byte integer gives the value 6075, but we know that the item id of rope is 3003. If we plug 0xBB (187) into the equation we get ((128^0)*(187))-(128^0)=186. We use 0 for x because 0xBB is the first byte and we count up from 0. Now, if we plug 0x17 (23) into the equation we get ((128^1)*(23))-(128^1)=2816. We use 1 for x here because it's the second byte but we count up from 0. Take the sum of both values and you get 186+2816=3002. The item id of rope is actually 3003, so we just need to add 1 to the sum.

pseudo-code:
Code:
int readValue(byte[] buffer, int position, int index = 0) {
    byte num1 = buffer[position];
    int num2 = ((128^index)*(num1))-(128^index);

    if (index == 0) {
        num2++;
    }

    if (num1 < 128) {
        return num2;
    }

    return num2 + readValue(buffer, position++, index++);
}

I'm not the best at formulas, so if anyone can write a better one to remove that first if-statement that would be great.
(128 ^ x) * (n - 1) and you could instead of doing recursively, use a while and after the loop is done you would sum 1 to the result, then you wouldn't have this if there.
 
Deserializing the integers should work like this:

Code:
let i = 0;
let result = 0;
let current;
do {
    current = buffer[i];
    let value = (current & 0b01111111) << (i++ * 7);
    result = result | value;
} while (current >= 0x80);
 
Last edited:
I wrote a parser for the new .dat format.

Check it out at https://github.com/TheSumm/datparserjs .
The protobuf .proto file can be found at https://github.com/TheSumm/datparserjs/blob/master/dat.proto, it can surely be improved as it's the first time I worked with protobuf.

I am not sure of the values fixedFrameGroup and id of FrameGroup are correct or what their purpose is.
Neither am I sure how the size of the sprite is stored.

Hello

using a script from this post I could extract two .proto files from the linux version of the client 11.

shared.proto
Code:
package tibia.protobuf.shared;


enum PLAYER_ACTION {
 PLAYER_ACTION_NONE = 0;
 PLAYER_ACTION_LOOK = 1;
 PLAYER_ACTION_USE = 2;
 PLAYER_ACTION_OPEN = 3;
 PLAYER_ACTION_AUTOWALK_HIGHLIGHT = 4;
}

enum ITEM_CATEGORY {
 ITEM_CATEGORY_ARMORS = 1;
 ITEM_CATEGORY_AMULETS = 2;
 ITEM_CATEGORY_BOOTS = 3;
 ITEM_CATEGORY_CONTAINERS = 4;
 ITEM_CATEGORY_DECORATION = 5;
 ITEM_CATEGORY_FOOD = 6;
 ITEM_CATEGORY_HELMETS_HATS = 7;
 ITEM_CATEGORY_LEGS = 8;
 ITEM_CATEGORY_OTHERS = 9;
 ITEM_CATEGORY_POTIONS = 10;
 ITEM_CATEGORY_RINGS = 11;
 ITEM_CATEGORY_RUNES = 12;
 ITEM_CATEGORY_SHIELDS = 13;
 ITEM_CATEGORY_TOOLS = 14;
 ITEM_CATEGORY_VALUABLES = 15;
 ITEM_CATEGORY_AMMUNITION = 16;
 ITEM_CATEGORY_AXES = 17;
 ITEM_CATEGORY_CLUBS = 18;
 ITEM_CATEGORY_DISTANCE_WEAPONS = 19;
 ITEM_CATEGORY_SWORDS = 20;
 ITEM_CATEGORY_WANDS_RODS = 21;
 ITEM_CATEGORY_PREMIUM_SCROLLS = 22;
 ITEM_CATEGORY_TIBIA_COINS = 23;
}

enum PLAYER_PROFESSION {
 PLAYER_PROFESSION_ANY = -1;
 PLAYER_PROFESSION_NONE = 0;
 PLAYER_PROFESSION_KNIGHT = 1;
 PLAYER_PROFESSION_PALADIN = 2;
 PLAYER_PROFESSION_SORCERER = 3;
 PLAYER_PROFESSION_DRUID = 4;
 PLAYER_PROFESSION_PROMOTED = 10;
}

enum ANIMATION_LOOP_TYPE {
 ANIMATION_LOOP_TYPE_PINGPONG = -1;
 ANIMATION_LOOP_TYPE_INFINITE = 0;
 ANIMATION_LOOP_TYPE_COUNTED = 1;
}

enum HOOK_TYPE {
 HOOK_TYPE_SOUTH = 1;
 HOOK_TYPE_EAST = 2;
}

appearances.proto
Code:
package tibia.protobuf.appearances;

import "shared.proto";


enum FIXED_FRAME_GROUP {
 FIXED_FRAME_GROUP_OUTFIT_IDLE = 0;
 FIXED_FRAME_GROUP_OUTFIT_MOVING = 1;
 FIXED_FRAME_GROUP_OBJECT_INITIAL = 2;
}
message Appearances {
 repeated Appearance object = 1;
 repeated Appearance outfit = 2;
 repeated Appearance effect = 3;
 repeated Appearance missile = 4;
}

message SpritePhase {
 optional uint32 duration_min = 1;
 optional uint32 duration_max = 2;
}

message SpriteAnimation {
 optional uint32 default_start_phase = 1;
 optional bool synchronized = 2;
 optional bool random_start_phase = 3;
 optional shared.ANIMATION_LOOP_TYPE loop_type = 4;
 optional uint32 loop_count = 5;
 repeated SpritePhase sprite_phase = 6;
}

message Box {
 optional uint32 x = 1;
 optional uint32 y = 2;
 optional uint32 width = 3;
 optional uint32 height = 4;
}

message SpriteInfo {
 optional uint32 pattern_width = 1;
 optional uint32 pattern_height = 2;
 optional uint32 pattern_depth = 3;
 optional uint32 layers = 4;
 repeated uint32 sprite_id = 5;
 optional uint32 bounding_square = 7;
 optional SpriteAnimation animation = 6;
 optional bool is_opaque = 8;
 repeated Box bounding_box_per_direction = 9;
}

message FrameGroup {
 optional FIXED_FRAME_GROUP fixed_frame_group = 1;
 optional uint32 id = 2;
 optional SpriteInfo sprite_info = 3;
}

message Appearance {
 optional uint32 id = 1;
 repeated FrameGroup frame_group = 2;
 optional AppearanceFlags flags = 3;
}

message AppearanceFlags {
 optional AppearanceFlagBank bank = 1;
 optional bool clip = 2;
 optional bool bottom = 3;
 optional bool top = 4;
 optional bool container = 5;
 optional bool cumulative = 6;
 optional bool usable = 7;
 optional bool forceuse = 8;
 optional bool multiuse = 9;
 optional AppearanceFlagWrite write = 10;
 optional AppearanceFlagWriteOnce write_once = 11;
 optional bool liquidpool = 12;
 optional bool unpass = 13;
 optional bool unmove = 14;
 optional bool unsight = 15;
 optional bool avoid = 16;
 optional bool no_movement_animation = 17;
 optional bool take = 18;
 optional bool liquidcontainer = 19;
 optional bool hang = 20;
 optional AppearanceFlagHook hook = 21;
 optional bool rotate = 22;
 optional AppearanceFlagLight light = 23;
 optional bool dont_hide = 24;
 optional bool translucent = 25;
 optional AppearanceFlagShift shift = 26;
 optional AppearanceFlagHeight height = 27;
 optional bool lying_object = 28;
 optional bool animate_always = 29;
 optional AppearanceFlagAutomap automap = 30;
 optional AppearanceFlagLenshelp lenshelp = 31;
 optional bool fullbank = 32;
 optional bool ignore_look = 33;
 optional AppearanceFlagClothes clothes = 34;
 optional AppearanceFlagDefaultAction default_action = 35;
 optional AppearanceFlagMarket market = 36;
 optional bool wrap = 37;
 optional bool unwrap = 38;
 optional bool topeffect = 39;
}

message AppearanceFlagBank {
 optional uint32 waypoints = 1;
}

message AppearanceFlagWrite {
 optional uint32 max_text_length = 1;
}

message AppearanceFlagWriteOnce {
 optional uint32 max_text_length_once = 1;
}

message AppearanceFlagLight {
 optional uint32 brightness = 1;
 optional uint32 color = 2;
}

message AppearanceFlagHeight {
 optional uint32 elevation = 1;
}

message AppearanceFlagShift {
 optional uint32 x = 1;
 optional uint32 y = 2;
}

message AppearanceFlagClothes {
 optional uint32 slot = 1;
}

message AppearanceFlagDefaultAction {
 optional shared.PLAYER_ACTION action = 1;
}

message AppearanceFlagMarket {
 optional shared.ITEM_CATEGORY category = 1;
 optional uint32 trade_as_object_id = 2;
 optional uint32 show_as_object_id = 3;
 optional string name = 4;
 repeated shared.PLAYER_PROFESSION restrict_to_profession = 5;
 optional uint32 minimum_level = 6;
}

message AppearanceFlagAutomap {
 optional uint32 color = 1;
}

message AppearanceFlagHook {
 optional shared.HOOK_TYPE direction = 1;
}

message AppearanceFlagLenshelp {
 optional uint32 id = 1;
}
 
Anyone is able to extract proto files from the newest client?

There were some changes, those posted above are not actual anymore and the mentioned script is unable to extract them from the newest client (gives only shared.proto and exits with errors).

Thanks.
 
Im very interested in this. This may lead to a tibia 11 item editor?

I want to up this thread.
 
Last edited by a moderator:
Im very interested in this. This may lead to a tibia 11 item editor?
Yeah, it would be very easy to make an item editor for Tibia 11 knowing these are just protobuf files. You can use Protod on the client to extract all of it's .proto files, then you can use Google's protoc executable (you can find an already compiled version under Releases) on those .proto files to generate code to use with C++, Go, Java, Python, Ruby, C#, Objective C, Javascript, or PHP. Just add the generated code to your project to easily parse and work with the protobuf files.
 
Yeah, it would be very easy to make an item editor for Tibia 11 knowing these are just protobuf files. You can use Protod on the client to extract all of it's .proto files, then you can use Google's protoc executable (you can find an already compiled version under Releases) on those .proto files to generate code to use with C++, Go, Java, Python, Ruby, C#, Objective C, Javascript, or PHP. Just add the generated code to your project to easily parse and work with the protobuf files.
Thanks for the pointing, will make some research

Thanks for the pointing, will make some research
I've been working with all those links you gave me. I already managed to get the *.dat from tibia 11 loaded my c# project.
My next step is to integrate with the sprite dump prohect is out there so i will be able to display all items information + image.

Next step after that is: How to integrate the items.otb part? I need to check the otb editors, maybe is not that hard.

Edit:

I have now a "working" solution in C#, whit a simple WindowsForm application, i do load .dat and sprites.

If someone is interested, i can upload it to github so you check the code and we all can start guessing how the sprites are managed. There are different sprites tipes and areas... iu still dont know all of them, and that will be needed to crop the sprite images correctly.
 
Last edited by a moderator:
Back
Top