# Tibia 11 .dat File Structure

Discussion in 'Discussion' started by Jo3Bingham, Oct 19, 2016.

1. ### Jo3BinghamWell-Known Member

Joined:
Mar 3, 2008
Messages:
975
413
4

Here you can see my progress in understanding the new .dat file structure. My 4th post (Update #3) contains a basically-complete breakdown of the structure. I still have a few flags to find, and some unknown values to figure out, but, for the most part, I have the whole structure decoded.

2. ### Tarek1337Well-Known Member

Joined:
Jun 9, 2012
Messages:
3,558
1,132
3
omg, this is nice. .

gj Jo3

Joined:
Jan 8, 2011
Messages:
193
327
0
Nice!

4. ### Jo3BinghamWell-Known Member

Joined:
Mar 3, 2008
Messages:
975
413
4
Update #4

Thanks to Daniel (blackd), the weird numbers (like with object ids, and some flags) has been solved.

If a value is greater-than-or-equal to 0x80 then there will be another value following it. If that value is greater-than-or-equal to 0x80 there will be another value following it. So on and so forth until the next value is less-than 0x80, but it should never be more than four values. Furthermore, the value of each byte can be calculated with the following formula: ((128^x)*(n))-(128^x) where x is the index of the current byte (0-based) and n is the value of the byte. Using recursion, and adding each calculated value, we can get the final value.

For example, the object id of rope is 0xBB 0x17 which, when read as a 2-byte integer gives the value 6075, but we know that the item id of rope is 3003. If we plug 0xBB (187) into the equation we get ((128^0)*(187))-(128^0)=186. We use 0 for x because 0xBB is the first byte and we count up from 0. Now, if we plug 0x17 (23) into the equation we get ((128^1)*(23))-(128^1)=2816. We use 1 for x here because it's the second byte but we count up from 0. Take the sum of both values and you get 186+2816=3002. The item id of rope is actually 3003, so we just need to add 1 to the sum.

pseudo-code:
Code (Text):
1.
2. int readValue(byte[] buffer, int position, int index = 0) {
3.     byte num1 = buffer[position];
4.     int num2 = ((128^index)*(num1))-(128^index);
5.
6.     if (index == 0) {
7.         num2++;
8.     }
9.
10.     if (num1 < 128) {
11.         return num2;
12.     }
13.
14.     return num2 + readValue(buffer, position++, index++);
15. }
16.
I'm not the best at formulas, so if anyone can write a better one to remove that first if-statement that would be great.

Joined:
May 27, 2007
Messages:
6,384
1,390
20
Sounds like protobuf.

Amiroslo likes this.
6. ### MkaloボーカロイドSupport Team

Joined:
Jun 1, 2011
Messages:
1,120
902
55
(128 ^ x) * (n - 1) and you could instead of doing recursively, use a while and after the loop is done you would sum 1 to the result, then you wouldn't have this if there.

Joined:
Jul 27, 2013
Messages:
421
280
2
8. ### Summ(\/)(;,,;)(\/) Y not?Staff MemberGlobal Moderator

Joined:
Oct 15, 2008
Messages:
4,167
1,065
9
Deserializing the integers should work like this:

Code (Text):
1.
2. let i = 0;
3. let result = 0;
4. let current;
5. do {
6.     current = buffer[i];
7.     let value = (current & 0b01111111) << (i++ * 7);
8.     result = result | value;
9. } while (current >= 0x80);
10.

Last edited: Oct 21, 2016
Tarek1337, MartyX and Mkalo like this.
9. ### Summ(\/)(;,,;)(\/) Y not?Staff MemberGlobal Moderator

Joined:
Oct 15, 2008
Messages:
4,167
1,065
9
Yamaken, Syntax, Ninja and 4 others like this.
10. ### godoyxdNoob

Joined:
Mar 6, 2008
Messages:
196
34
0
Hello

using a script from this post I could extract two .proto files from the linux version of the client 11.

shared.proto
Code (Text):
1. package tibia.protobuf.shared;
2.
3.
4. enum PLAYER_ACTION {
5.  PLAYER_ACTION_NONE = 0;
6.  PLAYER_ACTION_LOOK = 1;
7.  PLAYER_ACTION_USE = 2;
8.  PLAYER_ACTION_OPEN = 3;
9.  PLAYER_ACTION_AUTOWALK_HIGHLIGHT = 4;
10. }
11.
12. enum ITEM_CATEGORY {
13.  ITEM_CATEGORY_ARMORS = 1;
14.  ITEM_CATEGORY_AMULETS = 2;
15.  ITEM_CATEGORY_BOOTS = 3;
16.  ITEM_CATEGORY_CONTAINERS = 4;
17.  ITEM_CATEGORY_DECORATION = 5;
18.  ITEM_CATEGORY_FOOD = 6;
19.  ITEM_CATEGORY_HELMETS_HATS = 7;
20.  ITEM_CATEGORY_LEGS = 8;
21.  ITEM_CATEGORY_OTHERS = 9;
22.  ITEM_CATEGORY_POTIONS = 10;
23.  ITEM_CATEGORY_RINGS = 11;
24.  ITEM_CATEGORY_RUNES = 12;
25.  ITEM_CATEGORY_SHIELDS = 13;
26.  ITEM_CATEGORY_TOOLS = 14;
27.  ITEM_CATEGORY_VALUABLES = 15;
28.  ITEM_CATEGORY_AMMUNITION = 16;
29.  ITEM_CATEGORY_AXES = 17;
30.  ITEM_CATEGORY_CLUBS = 18;
31.  ITEM_CATEGORY_DISTANCE_WEAPONS = 19;
32.  ITEM_CATEGORY_SWORDS = 20;
33.  ITEM_CATEGORY_WANDS_RODS = 21;
35.  ITEM_CATEGORY_TIBIA_COINS = 23;
36. }
37.
38. enum PLAYER_PROFESSION {
39.  PLAYER_PROFESSION_ANY = -1;
40.  PLAYER_PROFESSION_NONE = 0;
41.  PLAYER_PROFESSION_KNIGHT = 1;
43.  PLAYER_PROFESSION_SORCERER = 3;
44.  PLAYER_PROFESSION_DRUID = 4;
45.  PLAYER_PROFESSION_PROMOTED = 10;
46. }
47.
48. enum ANIMATION_LOOP_TYPE {
49.  ANIMATION_LOOP_TYPE_PINGPONG = -1;
50.  ANIMATION_LOOP_TYPE_INFINITE = 0;
51.  ANIMATION_LOOP_TYPE_COUNTED = 1;
52. }
53.
54. enum HOOK_TYPE {
55.  HOOK_TYPE_SOUTH = 1;
56.  HOOK_TYPE_EAST = 2;
57. }
appearances.proto
Code (Text):
1. package tibia.protobuf.appearances;
2.
3. import "shared.proto";
4.
5.
6. enum FIXED_FRAME_GROUP {
7.  FIXED_FRAME_GROUP_OUTFIT_IDLE = 0;
8.  FIXED_FRAME_GROUP_OUTFIT_MOVING = 1;
9.  FIXED_FRAME_GROUP_OBJECT_INITIAL = 2;
10. }
11. message Appearances {
12.  repeated Appearance object = 1;
13.  repeated Appearance outfit = 2;
14.  repeated Appearance effect = 3;
15.  repeated Appearance missile = 4;
16. }
17.
18. message SpritePhase {
19.  optional uint32 duration_min = 1;
20.  optional uint32 duration_max = 2;
21. }
22.
23. message SpriteAnimation {
24.  optional uint32 default_start_phase = 1;
25.  optional bool synchronized = 2;
26.  optional bool random_start_phase = 3;
27.  optional shared.ANIMATION_LOOP_TYPE loop_type = 4;
28.  optional uint32 loop_count = 5;
29.  repeated SpritePhase sprite_phase = 6;
30. }
31.
32. message Box {
33.  optional uint32 x = 1;
34.  optional uint32 y = 2;
35.  optional uint32 width = 3;
36.  optional uint32 height = 4;
37. }
38.
39. message SpriteInfo {
40.  optional uint32 pattern_width = 1;
41.  optional uint32 pattern_height = 2;
42.  optional uint32 pattern_depth = 3;
43.  optional uint32 layers = 4;
44.  repeated uint32 sprite_id = 5;
45.  optional uint32 bounding_square = 7;
46.  optional SpriteAnimation animation = 6;
47.  optional bool is_opaque = 8;
48.  repeated Box bounding_box_per_direction = 9;
49. }
50.
51. message FrameGroup {
52.  optional FIXED_FRAME_GROUP fixed_frame_group = 1;
53.  optional uint32 id = 2;
54.  optional SpriteInfo sprite_info = 3;
55. }
56.
57. message Appearance {
58.  optional uint32 id = 1;
59.  repeated FrameGroup frame_group = 2;
60.  optional AppearanceFlags flags = 3;
61. }
62.
63. message AppearanceFlags {
64.  optional AppearanceFlagBank bank = 1;
65.  optional bool clip = 2;
66.  optional bool bottom = 3;
67.  optional bool top = 4;
68.  optional bool container = 5;
69.  optional bool cumulative = 6;
70.  optional bool usable = 7;
71.  optional bool forceuse = 8;
72.  optional bool multiuse = 9;
73.  optional AppearanceFlagWrite write = 10;
74.  optional AppearanceFlagWriteOnce write_once = 11;
75.  optional bool liquidpool = 12;
76.  optional bool unpass = 13;
77.  optional bool unmove = 14;
78.  optional bool unsight = 15;
79.  optional bool avoid = 16;
80.  optional bool no_movement_animation = 17;
81.  optional bool take = 18;
82.  optional bool liquidcontainer = 19;
83.  optional bool hang = 20;
84.  optional AppearanceFlagHook hook = 21;
85.  optional bool rotate = 22;
86.  optional AppearanceFlagLight light = 23;
87.  optional bool dont_hide = 24;
88.  optional bool translucent = 25;
89.  optional AppearanceFlagShift shift = 26;
90.  optional AppearanceFlagHeight height = 27;
91.  optional bool lying_object = 28;
92.  optional bool animate_always = 29;
93.  optional AppearanceFlagAutomap automap = 30;
94.  optional AppearanceFlagLenshelp lenshelp = 31;
95.  optional bool fullbank = 32;
96.  optional bool ignore_look = 33;
97.  optional AppearanceFlagClothes clothes = 34;
98.  optional AppearanceFlagDefaultAction default_action = 35;
99.  optional AppearanceFlagMarket market = 36;
100.  optional bool wrap = 37;
101.  optional bool unwrap = 38;
102.  optional bool topeffect = 39;
103. }
104.
105. message AppearanceFlagBank {
106.  optional uint32 waypoints = 1;
107. }
108.
109. message AppearanceFlagWrite {
110.  optional uint32 max_text_length = 1;
111. }
112.
113. message AppearanceFlagWriteOnce {
114.  optional uint32 max_text_length_once = 1;
115. }
116.
117. message AppearanceFlagLight {
118.  optional uint32 brightness = 1;
119.  optional uint32 color = 2;
120. }
121.
122. message AppearanceFlagHeight {
123.  optional uint32 elevation = 1;
124. }
125.
126. message AppearanceFlagShift {
127.  optional uint32 x = 1;
128.  optional uint32 y = 2;
129. }
130.
131. message AppearanceFlagClothes {
132.  optional uint32 slot = 1;
133. }
134.
135. message AppearanceFlagDefaultAction {
136.  optional shared.PLAYER_ACTION action = 1;
137. }
138.
139. message AppearanceFlagMarket {
140.  optional shared.ITEM_CATEGORY category = 1;
141.  optional uint32 trade_as_object_id = 2;
142.  optional uint32 show_as_object_id = 3;
143.  optional string name = 4;
144.  repeated shared.PLAYER_PROFESSION restrict_to_profession = 5;
145.  optional uint32 minimum_level = 6;
146. }
147.
148. message AppearanceFlagAutomap {
149.  optional uint32 color = 1;
150. }
151.
152. message AppearanceFlagHook {
153.  optional shared.HOOK_TYPE direction = 1;
154. }
155.
156. message AppearanceFlagLenshelp {
157.  optional uint32 id = 1;
158. }
159.

Jo3Bingham likes this.
11. ### Gold RayNew Member

Joined:
Jun 18, 2012
Messages:
8
3
0
Anyone is able to extract proto files from the newest client?

There were some changes, those posted above are not actual anymore and the mentioned script is unable to extract them from the newest client (gives only shared.proto and exits with errors).

Thanks.

12. ### elderaActive Member

Joined:
Oct 27, 2012
Messages:
157
39
0
Gold Ray likes this.
13. ### samco4x4 Developer.

Joined:
Jul 3, 2007
Messages:
879
180
6
Im very interested in this. This may lead to a tibia 11 item editor?

I want to up this thread.

Last edited by a moderator: Jun 13, 2018
14. ### Jo3BinghamWell-Known Member

Joined:
Mar 3, 2008
Messages:
975
413
4
Yeah, it would be very easy to make an item editor for Tibia 11 knowing these are just protobuf files. You can use Protod on the client to extract all of it's .proto files, then you can use Google's protoc executable (you can find an already compiled version under Releases) on those .proto files to generate code to use with C++, Go, Java, Python, Ruby, C#, Objective C, Javascript, or PHP. Just add the generated code to your project to easily parse and work with the protobuf files.

15. ### samco4x4 Developer.

Joined:
Jul 3, 2007
Messages:
879
180
6
Thanks for the pointing, will make some research

I've been working with all those links you gave me. I already managed to get the *.dat from tibia 11 loaded my c# project.
My next step is to integrate with the sprite dump prohect is out there so i will be able to display all items information + image.

Next step after that is: How to integrate the items.otb part? I need to check the otb editors, maybe is not that hard.

Edit:

I have now a "working" solution in C#, whit a simple WindowsForm application, i do load .dat and sprites.

If someone is interested, i can upload it to github so you check the code and we all can start guessing how the sprites are managed. There are different sprites tipes and areas... iu still dont know all of them, and that will be needed to crop the sprite images correctly.

Last edited by a moderator: Jun 17, 2018