Tofame
Advanced OT User
Hello everyone,
Im glad to share information about a revolutionary change, fully authored by me, which completely removes the need for .otb in OT projects. After analyzing .otb and .dat, I developed a method to rely only on .dat, ordered by server IDs, simplifying item handling dramatically -
The .otb format was originally introduced as a bridge between client IDs and server IDs. Allegedly, this came from old quirks with item removal/items being moved to different ids, every time new .dat was released, to confuse tfs developers so OTS wouldnt have working items, many years ago. My research and testing show:
In short: .otb is fully obsolete, and projects can safely move to .dat only.
Canary server had a similar idea, but they did not reorder .dat by server ID, as it would require a tool. This idea wasn't also easy to come up with, to begin with. Keep in mind that, we both use 'client ID', but my clientIDs are actually not equal to theirs. Also, in their method they had to patch thousands of lines in other systems to accommodate client-server ID differences. Imo my solution is better, but it's not like Canary's is worse. The amount of time I had to put into making the tool and even coming up with this idea balances things out. What matters that both me and Canary dont use
My solution:
[Q] Are IDs still shuffled inside the .dat, in the middle of it, to confuse everyone?
@zbizu answered me on discord:
[Q] A worry that I've received on discord:
My approach to server DOES NOT require changes in: scripts, .xml files, c++ files of TFS (other than .dat load method), which means everything is still backwards compatible, as no IDs are required to be changed across the codebase.
My approach to RME: I actually kept support for
Regarding client, as mentioned above, only SOME of the custom modules might be affected.
It's the fastest, flawless and most simple .otb deletion!
[Q] Why would I even support
So people can load maps with .dat/.otb that didnt go through a reorder of ids, meaning any RME with my commits can still open any map, no matter what files of items (ordered or not) a user has.
Basically, I wanted anyone who has either older .otb + .dat or new .dat to be able to load any kind of maps, without actually having to use different RMEs with different code.
Another benefit, that even I didn't predict, is that while I had some issues with my .dat loading in RME, users could load .otb and report to me in a more accurate way - whether it was due to my changes that something stopped working in RME, or it happens with .otb as well (meaning its not related to me) – so that was just as a fun fact, lol.
[Q] Is the process quick or is it difficult, or did it make it ‘harder’ to use engine that went full .dat compared to engines that still use .otb?
Yes, this process is extremely quickly and simple. Reordering on .dat is to be done just once, and afterwards, everything JUST keeps working. Simple. It’s black magic, until it’s not. Hopefully my explanations on the approach and how I’ve done it, after read maybe a few times, finally show that there are no flaws
[Q] Client is using Tibia.dat, but Server is using assets.dat. What?
I want to call the modified file assets.dat. The perfect solution would be to just modify OTC to look for
Why even rename? Reasons:
1. Avoids confusion when a person has Tibia.dat vs newName.dat, its easier to know if its actually reordered or not, which makes a big difference whether someone will have non-glitched client or w/e.
2. Redundant association to call a file 'Tibia' when it can store any type of items, outfits, effects.
3. Even Tibia doesnt call their file like that nowadays, they have sheets + appearances.dat
[Q] RME, what do I do, how do I map now?
If RME detects items.otb in its folder, it uses it. If it can’t find items.otb – either deleted or renamed, it simply load .dat. The path is already known since we give it when loading .spr… so no need to put any file in RME anymore
[Q] Can I use .otb?
The whole point of this, is to remove the need for .otb and item editor, at which I succeeded. The use cases that I predict should validate the need for .otb is only to load e.g. some datapacks and copy parts of the map, so...
Three variants:
[Q] Supported versions?
10.11-10.98. I don't have time to currently add 8.x+, but I wish to do so, plan is to add it in the next 6 months. It's simple, just not a priority for me. Maybe somebody else would do that in the meantime
[Q] I have custom items in my .dat, what now?
There is absolutely no difference, what anyone has done to their dat, whether it has 20k items or 100k items or any other number.
All that is needed is to just reorder the .dat by the tool to get the new dat.
Afterwards everything works - "Save' in OB builder will make items accessible in the server, map editor, client, all without even thinking of using item editor or anything else!
Im glad to share information about a revolutionary change, fully authored by me, which completely removes the need for .otb in OT projects. After analyzing .otb and .dat, I developed a method to rely only on .dat, ordered by server IDs, simplifying item handling dramatically -
.otb, using Item Editor app, converting sid<->sid <--- all of it is redundant.Why .otb is unnecessary
The .otb format was originally introduced as a bridge between client IDs and server IDs. Allegedly, this came from old quirks with item removal/items being moved to different ids, every time new .dat was released, to confuse tfs developers so OTS wouldnt have working items, many years ago. My research and testing show:
- .otb does not implement anything new; everything it contains is derivable from .dat.
- Most “custom” attributes in .otb are either calculated from .dat (like stack order) or deprecated (from 7.x times when items.xml didnt exist).
- Only minor attributes like allowDistRead were .otb-specific, and even those are mostly covered by items.xml.
- The original client-server ID mapping is no longer necessary because the new .dat can be reordered by server ID, making cid == sid.
In short: .otb is fully obsolete, and projects can safely move to .dat only.
My approach vs Canary server
Canary server had a similar idea, but they did not reorder .dat by server ID, as it would require a tool. This idea wasn't also easy to come up with, to begin with. Keep in mind that, we both use 'client ID', but my clientIDs are actually not equal to theirs. Also, in their method they had to patch thousands of lines in other systems to accommodate client-server ID differences. Imo my solution is better, but it's not like Canary's is worse. The amount of time I had to put into making the tool and even coming up with this idea balances things out. What matters that both me and Canary dont use
.otb anymore!!My solution:
- I built a custom tool to reorder .dat by server IDs, making client IDs match server IDs directly.
- This eliminates the need for .otb and for any extensive downstream code changes.
- After conversion, .otb becomes fully obsolete, and everything works with a single ID per item.
Technical Notes on how to make such a tool
- Read both old .otb and .dat initially, then while looping through otb items you will be getting each dat Item by server id and outputting it. The final result will be a .dat that is ordered by serverids!
- [NOTE] After reordering, cid == sid across all items.
- [NOTE] Remember that .otb had uint32_t flags, while .dat has its attributes saved byte after byte.
[Q] Are IDs still shuffled inside the .dat, in the middle of it, to confuse everyone?
@zbizu answered me on discord:
Code:
they no longer do
it was up until 8.x times
they no longer do it since 9 something[Q] A worry that I've received on discord:
otb never was removed due to the strict rules in place for the entire opentibia community, dont do stuff that can break compatibility with thousand versionsMy approach to server DOES NOT require changes in: scripts, .xml files, c++ files of TFS (other than .dat load method), which means everything is still backwards compatible, as no IDs are required to be changed across the codebase.
My approach to RME: I actually kept support for
.otb loading, which means with my commits both .otb and/or_only .dat can be loaded!Regarding client, as mentioned above, only SOME of the custom modules might be affected.
It's the fastest, flawless and most simple .otb deletion!
[Q] Why would I even support
.otb in RME, why didn't I just remove that code from RME like I did for the server?So people can load maps with .dat/.otb that didnt go through a reorder of ids, meaning any RME with my commits can still open any map, no matter what files of items (ordered or not) a user has.
Basically, I wanted anyone who has either older .otb + .dat or new .dat to be able to load any kind of maps, without actually having to use different RMEs with different code.
Another benefit, that even I didn't predict, is that while I had some issues with my .dat loading in RME, users could load .otb and report to me in a more accurate way - whether it was due to my changes that something stopped working in RME, or it happens with .otb as well (meaning its not related to me) – so that was just as a fun fact, lol.
[Q] Is the process quick or is it difficult, or did it make it ‘harder’ to use engine that went full .dat compared to engines that still use .otb?
Yes, this process is extremely quickly and simple. Reordering on .dat is to be done just once, and afterwards, everything JUST keeps working. Simple. It’s black magic, until it’s not. Hopefully my explanations on the approach and how I’ve done it, after read maybe a few times, finally show that there are no flaws
[Q] Client is using Tibia.dat, but Server is using assets.dat. What?
I want to call the modified file assets.dat. The perfect solution would be to just modify OTC to look for
assets.dat in the folder, but since that modification is not done (despite probably being like 3 lines in c++), the user is expected to take assets.dat, rename it to Tibia.dat and put in the client.Why even rename? Reasons:
1. Avoids confusion when a person has Tibia.dat vs newName.dat, its easier to know if its actually reordered or not, which makes a big difference whether someone will have non-glitched client or w/e.
2. Redundant association to call a file 'Tibia' when it can store any type of items, outfits, effects.
3. Even Tibia doesnt call their file like that nowadays, they have sheets + appearances.dat
[Q] RME, what do I do, how do I map now?
If RME detects items.otb in its folder, it uses it. If it can’t find items.otb – either deleted or renamed, it simply load .dat. The path is already known since we give it when loading .spr… so no need to put any file in RME anymore
[Q] Can I use .otb?
The whole point of this, is to remove the need for .otb and item editor, at which I succeeded. The use cases that I predict should validate the need for .otb is only to load e.g. some datapacks and copy parts of the map, so...
Three variants:
- old items.otb and old .dat
- new .dat AND new items.otb
- just .dat (like I encourage anyone to do)
[Q] Supported versions?
10.11-10.98. I don't have time to currently add 8.x+, but I wish to do so, plan is to add it in the next 6 months. It's simple, just not a priority for me. Maybe somebody else would do that in the meantime
[Q] I have custom items in my .dat, what now?
There is absolutely no difference, what anyone has done to their dat, whether it has 20k items or 100k items or any other number.
All that is needed is to just reorder the .dat by the tool to get the new dat.
Afterwards everything works - "Save' in OB builder will make items accessible in the server, map editor, client, all without even thinking of using item editor or anything else!
Last edited by a moderator:
reorder is needed no matter what imo
