• There is NO official Otland's Discord server and NO official Otland's server list. The Otland's Staff does not manage any Discord server or server list. Moderators or administrator of any Discord server or server lists have NO connection to the Otland's Staff. Do not get scammed!

OpenTibia OTClient that unpack map.otbm to .png files

Gesior.pl

Mega Noob&LOL 2012
Senator
Joined
Sep 18, 2007
Messages
2,955
Solutions
98
Reaction score
3,350
Location
Poland
GitHub
gesior
This is modified OTClient that generates .png images of OTS .otbm map!
Each image show 8x8 tiles, 256x256 px, you can easily use it with Leaflet Maps [like GoogleMaps] to show whole map on www with zoom in/out.
Example image:
ai5ssKdqZ.png


I will give full description, when I fix all bugs.

On my PC it generates around 40 images per second on 1 core and up to 200 per second on all cores (4x4.0 GHz). I run 16 threads, so it faster skips empty areas [areas without tiles].

Without SSD disc, it will kill your machine and take around 3213 days to generate RL map images, so don't even try :)

Short instruction:
1. Compile, copy Tibia.spr, Tibia.dat, items.otb, map.otbm, run client.
2. Type command:

Code:
prepareClient(1076, '/things/1076/items.otb', '/map.otbm')
It will freez client for a while and then show message like:
Example generator of whole map: generateMap(25, 45, 0, 555, 699, 15, 4) [last 4 = 4 threads to generate]
Positions in this command 'example' are GENERATED FOR YOUR MAP FILE. These are minimum and maximum x, y, z of all tiles. Use it to skip all empty areas (around 95% of RL map!) and save a lot of time.
3. Type command:
Code:
generateMap(25, 45, 0, 555, 699, 15, 4)
version for one core generation [no problem with client crashes]:
Code:
generateMap(25, 45, 0, 555, 699, 15, 1)

4. DONE!

Your map images will appear in C:\USERS\YOUR_USER_NAME\otclient\map\

USERS
- your Windows language 'users' (like 'Uzytkownicy' in polish windows 7)
YOUR_USER_NAME - your Windows login


Full instruction how to use you can find there:
https://github.com/gesior/otclient/blob/master/README.md

Source code:
https://github.com/gesior/otclient

All changes required (only 17!):
https://github.com/gesior/otclient/commit/6eee8b9673205312cf1b21332067037e7ba81010

Most of logic is in LUA (thread management):
https://github.com/gesior/otclient/blob/master/otclientrc.lua


------------------------------------------------------------
I also got code for Leaflet map [PHP generators]:

- generate zoom levels images (zoom in/out that save a lot of transfer)
- show lower floors when you are on floor over ground (lower floors are 'shaded')
- minimize size of images [less then 10GB for 140 MB RL map .otbm]
- JavaScript class that contains functions like: Map.setCenter({x=23323,y=34332,z=6}) [in game position]
- JavaScript class (+LUA for TFS 1.0) to STREAM ALL PLAYERS [animated move, ~2 seconds delay] on Leaflet map - tested with RL map with 900 players online, no lag!
- JavaScript class for stream that let you search for player by name and 'follow' (map moves when player moves in game)

BUT I DON'T HAVE TIME TO DESCRIBE HOW TO CONFIGURE THEM ALL! I WILL TRY NEXT WEEKEND!
 
With this client you can also generate one big image of some part of map.
C++ function:
Code:
drawMap(std::string fileName, int sx, int sy, int sz, int size)
Example command to execute in Client Terminal, generate some town, give top-left position of town and it's size in tiles (generates only rectangles):
Code:
g_map.drawMap('thais_map.png', 32564, 33453, 7, 100)
It will generate 100 x 100 tiles image (3200x3200 pixels). Just set your town positions and file name.
 
so you can actually use it like map tracker?

can't edit posts ;/
split it to events like my map generator in mystera datapack so you will avoid "killing the machine"
 
Last edited by a moderator:
%homepath%/otclient/map
Windows users need instruction.
Linux users know that it's in '/home/username/otclient/map'.
I got no idea where are these folders on Mac and Android.
so you can actually use it like map tracker?
I think that map trackers track map in-game to .otbm file. My client version open .otbm and generate .png images of it, so I don't think so.
can't edit posts ;/
split it to events like my map generator in mystera datapack so you will avoid "killing the machine"
It's split to events, but on almost every PC it will generate files fast then normal HDD can write files, so it will freez Windows (problem with access to too many files per second).
.... PHP scripts that convert these .png images to format of Leaflet Map uses HDD even more, but PHP script you can run on dedic like every other website (most admins won't install Desktop linux on dedic, just to generate images in client with GUI).
 
ah ok
add option to use jpg format, it will save some performance I guess

btw.
http://tibia.wikia.com/wiki/Mapper
saw similiar project to this, it was called tibia earth, you may find some information useful to compress sprites on screen or something
 
Last edited by a moderator:
btw.
http://tibia.wikia.com/wiki/Mapper
saw similiar project to this, it was called tibia earth, you may find some information useful to compress sprites on screen or something
I know that few people somehow generated tibia map before (I did it 2-3 years ago with pure PHP script), but no one shared code for this. Probably these codes were very slow (like my PHP implementation).

PNG is supported by OTClient by default. I don't know if JPG compression is faster then PNG, but I'm sure it's lossy. After unpacking PHP script will have to edit all images 3-4 times. It can't be done with format that lose 20% of quality every 'save'.
 
@Gesior.pl Is there any change that the webgui is going to be released? (Like screenshots above?)

03ea5ee7d8.jpg


Using ORTS map & items.otb.
 
Last edited by a moderator:
03ea5ee7d8.jpg


Using ORTS map & items.otb.
OTClient compiled on windows is 32bit application and it can use maximum 2 GB ram. Not enough for ORTS map.
To unpack bigger map you need 64 bit libraries (idk where you can get all libraries for windows OTclient, on linux all is automatic).

I unpacked ORTS some time ago, but to do this I installed VMWare Player (virtual machine for windows) and on it I installed Ubuntu Desktop.
I set 6.5 GB ram to virtual machine (of 8GB) and unpacked images on it. I unpacked ORTS map on 64 bit linux in less then 2 hours.

I'm going to release these PHP and JS scripts [website view], but I got no time to describe them. I'm working on my friend's OTS engine (detect and ban bots automatically).

I unpacked RL map 7.4 on windows [1.5GB ram] in 3498 seconds on 4 cores (4x4GHz). Then converted ~450.000 PNG images to map format with 8 'levels of zoom' in 5-6 hours using 1-4 cores.
I wrote short instruction how to use scripts and in what order:
PHP:
1. Put generated map folders in 'map1Unpacked'
2. Run: 1removeEmptyFolders.php
3. Copy folders [floors] '7' and '15' from 'map1Unpacked' to 'map2Composed'
4. Run: 2composeFloors.php
5. Run: 3shader.php
5. Copy or rename 'map3Shaded' to 'map4Tiled/12'
6. Run: 4tileGenerator.php
7. Run: 5compressor.php
but I want make it automatic with JS script. You just open page, click 'start' and watch 'progress'.
 
OTClient compiled on windows is 32bit application and it can use maximum 2 GB ram. Not enough for ORTS map.
To unpack bigger map you need 64 bit libraries (idk where you can get all libraries for windows OTclient, on linux all is automatic).

I unpacked ORTS some time ago, but to do this I installed VMWare Player (virtual machine for windows) and on it I installed Ubuntu Desktop.
I set 6.5 GB ram to virtual machine (of 8GB) and unpacked images on it. I unpacked ORTS map on 64 bit linux in less then 2 hours.

I'm going to release these PHP and JS scripts [website view], but I got no time to describe them. I'm working on my friend's OTS engine (detect and ban bots automatically).

I unpacked RL map 7.4 on windows [1.5GB ram] in 3498 seconds on 4 cores (4x4GHz). Then converted ~450.000 PNG images to map format with 8 'levels of zoom' in 5-6 hours using 1-4 cores.
I wrote short instruction how to use scripts and in what order:
PHP:
1. Put generated map folders in 'map1Unpacked'
2. Run: 1removeEmptyFolders.php
3. Copy folders [floors] '7' and '15' from 'map1Unpacked' to 'map2Composed'
4. Run: 2composeFloors.php
5. Run: 3shader.php
5. Copy or rename 'map3Shaded' to 'map4Tiled/12'
6. Run: 4tileGenerator.php
7. Run: 5compressor.php
but I want make it automatic with JS script. You just open page, click 'start' and watch 'progress'.

Thanks for the reply, i'm going to use MinGW64 , lets see if i can get it working with that :)

I want to use the webgui just to display houses on website (So players can have an ingame view of how it looks like ingame, the can check all floors to).
bassicly replacing this.
1c6e5649e7.png


cant edit post above.

Where can i find the files you are speaking of to get webgui working?
 
Last edited by a moderator:
http://skalski.at/fileshosting/down...name=generate_leaflet_tiles_version_alpha.zip
In .zip are generators and simple view script with stream script for TFS 1.1 (LUA).

In map.js you got access to some map manipulation based on tibia in-game positions.
There is also config.
line 20 is relative path from which it should load images:
PHP:
mapTilesLevelsLayers[i] = L.tileLayer('../rlmap/levels/{z}/' + i + '/{x}/' + i + '_{x}_{y}.jpg', {
at end of file is startup config:
PHP:
Game.init({x: -32222.72, y: 32358.4, z: 7});
map.setZoom(12);
In alpha version 'x' parameter must be negative. I will fix it before release.

Of course.. I can't edit my post too.

In zip file are files with name like:
3shader_c1.php // core 1, for($z = 0; $z <= 3; $z++)
3shader_c2.php // core 2, for($z = 4; $z <= 7; $z++)
This is first version of parallelization, but you can use version without parallelization to not make too many mistakes in config of scripts.

In some files there is positions config in code. Like:
PHP:
    for($x = 3800; $x <= 5250; $x++)
    {
        for($y = 3800; $y <= 5250; $y++)
        {
It will convert only images with positions 30400 to 42000 (get these numbers and multiply by 8 [8 tiles per image]).

For ORTS map processing you will need 100-150GB of space on HDD (in final version I will delete files that won't be used anymore after each step - some steps need images from 2 previous steps!)
 
Last edited by a moderator:
Of course.. I can't edit my post too.

In zip file are files with name like:
3shader_c1.php // core 1, for($z = 0; $z <= 3; $z++)
3shader_c2.php // core 2, for($z = 4; $z <= 7; $z++)
This is first version of parallelization, but you can use version without parallelization to not make too many mistakes in config of scripts.

In some files there is positions config in code. Like:
PHP:
    for($x = 3800; $x <= 5250; $x++)
    {
        for($y = 3800; $y <= 5250; $y++)
        {
It will convert only images with positions 30400 to 42000 (get these numbers and multiply by 8 [8 tiles per image]).

For ORTS map processing you will need 100-150GB of space on HDD (in final version I will delete files that won't be used anymore after each step - some steps need images from 2 previous steps!)

Oke, i took your advice and switched over to linux. Altought i'm not using SSD. It's generating the map :) going to let it run whole night.
f3a036cc94.jpg


I'll try the scripts tomorrow as soon as i have all the images.
 
Oke, i took your advice and switched over to linux. Altought i'm not using SSD. It's generating the map :) going to let it run whole night.
f3a036cc94.jpg


I'll try the scripts tomorrow as soon as i have all the images.
Multi-threading is not ready yet (and idk if I will have time to fix it). I had few client crashes while I was generating 7.4 rl map. To reduce chance of crash I changed in file:
https://github.com/gesior/otclient/blob/master/otclientrc.lua
Lines:
PHP:
local areaSizeY = 25
local areaSizeX = 25
to:
PHP:
local areaSizeY = 100
local areaSizeX = 100
It changed number of images generated by thread from 625 to 10000. It does not affect speed of images generation. It just change progress messages (and too high value could send 'all map images' to 1 thread), but for RL map even 100.000 images per thread aren't too much.
I think that crash is when 2 threads try to read 1 tile (to generate image thread have to read tiles of 'image' +1 x/y [because some items have size 64x64 and are visible on positions - 1x/y: walls, stairs..] and that +1 x/y position is tile of other image).
Some solutions for it would be other algorithm of function threadsManager() that send different floor to each thread, so they cant read same tile.
 
Multi-threading is not ready yet (and idk if I will have time to fix it). I had few client crashes while I was generating 7.4 rl map. To reduce chance of crash I changed in file:
https://github.com/gesior/otclient/blob/master/otclientrc.lua
Lines:
PHP:
local areaSizeY = 25
local areaSizeX = 25
to:
PHP:
local areaSizeY = 100
local areaSizeX = 100
It changed number of images generated by thread from 625 to 10000. It does not affect speed of images generation. It just change progress messages (and too high value could send 'all map images' to 1 thread), but for RL map even 100.000 images per thread aren't too much.
I think that crash is when 2 threads try to read 1 tile (to generate image thread have to read tiles of 'image' +1 x/y [because some items have size 64x64 and are visible on positions - 1x/y: walls, stairs..] and that +1 x/y position is tile of other image).
Some solutions for it would be other algorithm of function threadsManager() that send different floor to each thread, so they cant read same tile.

Did what u said, for some reason i cant generate full map. Using code for full map generating but it only makes 16 images :p

830a834f5c.jpg
 
The crashes most likely occur due to the refcounter of Image objects being non-atomic, thus causing a data race when multiple threads access cached sprites. I have a patch almost ready, I'll create a PR on github later today or tomorrow when I get the time to clean it up.
 
Back
Top