OpenTibia OTClient that unpack map.otbm to .png files - v4.0

[Gesior.pl]

What is new in version 4.0?

• website view with NPCs and monsters search
• configurable town names and positions - big names visible on map
• possibility to link NPCs and monsters ex. mymap.com/#npc,Xodet , mymap.com/#monster,Dragon+Lord
• scripts to run 'zoom levels' and 'compression' on 16 cores on windows (for linux it was in version 3.0)
• house images generator - generate images of houses that you can use on 'houses' page in acc. maker
• minimap generator - you can generate full minimap and use it with any OTClient
• sources updated for new linuxes (new openssl) and newest Visual Studio project from edubart/otclient (vcpkg)

Github release with 64-bit .exe:

Release Map generator v4 - Windows x64 · gesior/otclient_mapgen

Open Tibia map images generator [for LeafletJS]. Loads .otbm file and generate .png images 256x256 px - Release Map generator v4 - Windows x64 · gesior/otclient_mapgen

github.com

What is this?
It's automatic map image generator [for website view 'like GoogleMaps'].
It generates OTS map .png images (each 8x8 tiles).

-- BINARY (.EXE) ATTACHED! YOU DO NOT HAVE TO COMPILE! --
To run binary you may need to install Microsoft VS 2019 Redist x64 first:

https://aka.ms/vs/16/release/vc_redist.x64.exe

It contains configured JavaScript and HTML to view map on website like this:


How to use?
1. Download otclient_mapgen from GitHub:
GitHub - gesior/otclient_mapgen: Open Tibia map images generator [good for Leaflets]. Loads .otbm file and generate .png images 256x256 px

and compile it like normal OTClient:
Compiling on Windows · edubart/otclient Wiki · GitHub
Compiling on Linux · edubart/otclient Wiki · GitHub
Compiling on Mac OS X · edubart/otclient Wiki · GitHub
Compiling for Android · edubart/otclient Wiki · GitHub

OR DOWNLOAD BINARY (.EXE) ATTACHED

2. Copy your server client .spr and .dat files to OTClient folder: data/things/HERE_PROTOCOL_VERSION/

3. Copy your server data/items/items.otb file to OTClient folder: data/things/HERE_PROTOCOL_VERSION/

4. Copy your server data/world/MAP_NAME.otbm file to OTClient folder: data/map.otbm

5. Run otclient.exe

6. Type in client terminal command like:

prepareClient(1076, '/things/1076/items.otb', '/map.otbm', 8, 5)

• with you client protocol version,
• valid paths to items.otb and map.otbm,
• number of threads to run (in example it's 8),
• number of map parts (OTC will split map to parts to reduce maximum RAM usage)

[Paths are relational, so start path with '/' to start in folder 'data', access to other folders in computer is blocked]

7. OTClient will show in 10-50 seconds (it will freez client, do not close it, just wait) something like:

Now just type (lower levels shadow 30%)
ALL PARTS OF MAP:
generateMap('all', 30)
ONLY PARTS 2 AND 3 OF MAP:
generateMap({2, 3}, 30)

8. Type in client terminal command:

generateMap('all', 30)

OTClient will report progress in terminal.

9. Your map images will appear in your system 'user' directory

%HOMEPATH%/otclient - Windows, open it in folder explorer
$({HOME}/.otclient - Linux, 'cd' to it, this folder is invisible, but you can navigate to it

10. Copy folder map from folder metioned in previous step to folder website_and_php_files of otclient_mapgen.

--- NEXT STEPS REQUIRE PHP INSTALLED IN SYSTEM ---

--- LINUX: FOR GENERATION TIME, SET FOLDER website_and_php_files RIGHTS TO 777 ---

11. Linux

1. Execute (in terminal):
   sh linux_run_all.sh

11. Windows
1. If you got Git BASH installed

1. Execute (in system terminal):
   ./windows_run_1_pre_tile_generator.sh
   
   
2. Execute:
   ./windows_run_2_tile_generator.sh
   
   Watch CPU usage (by php.exe). Wait until it finish.
   
   
3. Execute:
   ./windows_run_3_pre_compress.sh
   
   
4. Execute:
   ./windows_run_4_compress.sh
   
   Watch CPU usage (by php.exe). Wait until it finish.

2. If you do not have Git BASH

1. Execute (in system terminal):
   php 1_pre_tile_generator.php
   
   
2. Execute (command parameter is map 'floor'):
   • You can execute these commands in any order.
   • You can open few terminals and run few commands at once to generate it faster (use all CPU cores).
     
     php 2_tile_generator.php 0
     php 2_tile_generator.php 1
     php 2_tile_generator.php 2
     php 2_tile_generator.php 3
     php 2_tile_generator.php 4
     php 2_tile_generator.php 5
     php 2_tile_generator.php 6
     php 2_tile_generator.php 7
     php 2_tile_generator.php 8
     php 2_tile_generator.php 9
     php 2_tile_generator.php 10
     php 2_tile_generator.php 11
     php 2_tile_generator.php 12
     php 2_tile_generator.php 13
     php 2_tile_generator.php 14
     php 2_tile_generator.php 15
     php 2_tile_generator.php 16
     
     
3. Execute:
   php 3_pre_compress.php
   
   
4. Execute (command parameter is map 'floor'):
   • You can set compression quality in file 4_compress.php (line: $quality = 80
   • You can execute these commands in any order.
   • You can open few terminals and run few commands at once to generate it faster (use all CPU cores).
     
     php 4_compress.php 0
     php 4_compress.php 1
     php 4_compress.php 2
     php 4_compress.php 3
     php 4_compress.php 4
     php 4_compress.php 5
     php 4_compress.php 6
     php 4_compress.php 7
     php 4_compress.php 8
     php 4_compress.php 9
     php 4_compress.php 10
     php 4_compress.php 11
     php 4_compress.php 12
     php 4_compress.php 13
     php 4_compress.php 14
     php 4_compress.php 15
     php 4_compress.php 16

12. Move folder website_and_php_files/map_viewer to your webserver - it contains all images and website scripts to view them (Leaflet Map).

13. Configure map_viewer. Config is at start of map.js file:
- You can keep images on other webserver. Set imagesUrl to URL of this server like: 'http://myserver.com/map_images/'

imagesUrl: 'map/', // URL to folder with 'zoom levels' (folders with names 0-16)
imagesExtension: '.jpg',
mapName: 'RL MAP?',
startPosition: {x: 1000, y: 1000, z: 7},
startZoom: 14,
minZoom: 4,
maxZoom: 18, // maximum zoom with full quality is 16

GENERATE HOUSE IMAGES

1. Do steps 1-6 from How to use? instruction (above).
2. Type in client terminal command:
   generateHouses(30, false)
   • 30 - shadow percent for tiles around house (they will be darker)
   • false - load whole map at once, requires much RAM, set it to true to load map by parts, it's much slower, but will use little amount of RAM to generate house images
3. It will freez client for house generation time! Don't close it. If you want watch progress run OTClient by windows/linux console (not by clicking .exe):
   ./otclient
   'Freezed' client will show messages about generated houses in system console.
4. Your house images will appear in your system 'user' directory:
   %HOMEPATH%/otclient/house/ - Windows, open it in folder explorer
   ${HOME}/.otclient/house/ - Linux, 'cd' to it, this folder is invisible, but you can navigate to it

Houses with few floors generate few images with 'floor level' in file name, but all floors images have same size and top-left corner position, so you can show it on website with +/- button to switch floors.
Example house image - tiles outside house are 30% darker:


GENERATE FULL MINIMAP FOR OTCLIENT

Most of released OTClient .exe files are 32-bit and they got limit around 2 GB RAM. It means that you cannot generate minimap from .otbm bigger than ~60MB.
Attached binary is 64-bit version of OTClient. It can load any size .otbm into RAM. To export full minimap, you got to load whole map.

1. Do steps 1-6 from How to use? instruction (above).
2. Type in client terminal command:
   saveMinimap('minimap.otmm')
3. Your minimap will appear in your system 'user' directory:
   %HOMEPATH%/otclient/ - Windows, open it in folder explorer
   ${HOME}/.otclient/ - Linux, 'cd' to it, this folder is invisible, but you can navigate to it
4. Copy it to folder data in your normal OTClient. It must be closed when you place there new minimap!
5. Run your OTClient, login to game, you should see full minimap.

 

[Raikou]

This looks fancy, I'll be trying it out right now

Post automatically merged: Jun 14, 2020

Yeap as expected can't get it to work for some reason, i got php but it doesnt understand php, i got git bash but it's not reacting.

or via powershell

But it does nothing.

Than last method was do it via de PHP version, but suddenly my computer does not understand PHP

Which is funny since it actually runs a webserver using PHP. so yeap I tried but failed miserably.

 

[Gesior.pl]

@Raikou
1. In Git BASH you must 'run' things in linux style
1.1 Type name of 'known' (read point 2) binary file that you want to execute:

php 1_pre_tile_generator.php

/bin/bash windows_run_1_pre_tile_generator.sh

(/bin/bash is linux style path to BASH, but it's added/emulated by Git BASH and should work)

1.2 or start it with ./ and let system determine what program it should start - and pass given file as parameter, so it must be:

./windows_run_1_pre_tile_generator.sh

It should execute /bin/bash by itself, because in first line of file is: #!/bin/bash


2. Windows knows in console only programs listed in operating system 'Path'.
To edit it:
1. Open 'My Computer'
2. Click right mouse button on empty space and select last listed thing 'Preferences' / 'Attributes' (i dont know translation)
3. On left side click 'advanced system settings'.
4. Select 'Advanced' tab.
5. Click 'Environment variables'.
6. Find system variable 'Path' and edit it.
7. Add path to your folder with PHP (in my case it was C:\xampp\php\php.exe, so I added C:\xampp\php )

I don't know when Windows reloads that variable. Maybe running new console window will make it work. Maybe it will require user logout-login or even system restart.

Steps 1-3:


Steps 3-7:

 

[Raikou]

@Raikou
1. In Git BASH you must 'run' things in linux style
1.1 Type name of 'known' (read point 2) binary file that you want to execute:

php 1_pre_tile_generator.php

/bin/bash windows_run_1_pre_tile_generator.sh

(/bin/bash is linux style path to BASH, but it's added/emulated by Git BASH and should work)

1.2 or start it with ./ and let system determine what program it should start - and pass given file as parameter, so it must be:

./windows_run_1_pre_tile_generator.sh

It should execute /bin/bash by itself, because in first line of file is: #!/bin/bash


2. Windows knows in console only programs listed in operating system 'Path'.
To edit it:
1. Open 'My Computer'
2. Click right mouse button on empty space and select last listed thing 'Preferences' / 'Attributes' (i dont know translation)
3. On left side click 'advanced system settings'.
4. Select 'Advanced' tab.
5. Click 'Environment variables'.
6. Find system variable 'Path' and edit it.
7. Add path to your folder with PHP (in my case it was C:\xampp\php\php.exe, so I added C:\xampp\php )

I don't know when Windows reloads that variable. Maybe running new console window will make it work. Maybe it will require user logout-login or even system restart.

Steps 1-3:
View attachment 46571

Steps 3-7:
View attachment 46572

Thanks, that was very helpful especially the part where you tell how to put php as a system variable because that's why my php wasn't reacting in the console. I`m proceeding now, I'll keep you updated how things go

EDIT~
Got it to work, it shows the whole map now for a peek:

Spoiler: Dolport Challenges

Dolport Challenges Map

I do see it can also load monsters, trying to figure out if it's for in the future or just disabled.

 

[Gesior.pl]

Thanks, that was very helpful especially the part where you tell how to put php as a system variable because that's why my php wasn't reacting in the console. I`m proceeding now, I'll keep you updated how things go

EDIT~
Got it to work, it shows the whole map now for a peek:

Spoiler: Dolport Challenges

Dolport Challenges Map

I do see it can also load monsters, trying to figure out if it's for in the future or just disabled.

You got to edit file website_and_php_files/map_viewer/creaturesData.php
There are configurable paths to files required to make it work:

// path to folder with monster, npc and spawns data
$dataPath = '/home/ots/data/';
$monstersDir = 'monster/';
$npcDir = 'npc/';
$spawnsFile = 'world/world-spawn.xml';

If in this folder appeared file creaturesData.cache.json - remove it. It blocks reloading monsters list.

 

[Raikou]

You got to edit file website_and_php_files/map_viewer/creaturesData.php
There are configurable paths to files required to make it work:

// path to folder with monster, npc and spawns data
$dataPath = '/home/ots/data/';
$monstersDir = 'monster/';
$npcDir = 'npc/';
$spawnsFile = 'world/world-spawn.xml';

If in this folder appeared file creaturesData.cache.json - remove it. It blocks reloading monsters list.

It doesn't seem to work if the server is on another drive? (webserver is on C, server is on E)

// path to folder with monster, npc and spawns data
$dataPath = 'E:/TibiaOT/Dolport Server/data/';
$monstersDir = 'E:/TibiaOT/Dolport Server/data/monster/';
$npcDir = 'E:/TibiaOT/Dolport Server/data/npc/';
$spawnsFile = 'E:/TibiaOT/Dolport Server/data/world/Dolport-spawn.xml';

 

[Gesior.pl]

It doesn't seem to work if the server is on another drive? (webserver is on C, server is on E)

// path to folder with monster, npc and spawns data
$dataPath = 'E:/TibiaOT/Dolport Server/data/';
$monstersDir = 'E:/TibiaOT/Dolport Server/data/monster/';
$npcDir = 'E:/TibiaOT/Dolport Server/data/npc/';
$spawnsFile = 'E:/TibiaOT/Dolport Server/data/world/Dolport-spawn.xml';

Only possible issue is right to access to that folder. Maybe webserver is not allowed to read it.
First try to copy OTS to disc C.

 

[Raikou]

Sorry for my n00bness in this, but I put the otserver in the same folder now.

I delete the cache file and refresh the page, but sadly not working yet.
(I really start to feel dumb now)

// path to folder with monster, npc and spawns data
$dataPath = 'data/';
$monstersDir = "data/monster/";
$npcDir = 'data/npc/';
$spawnsFile = 'data/world/Dolport-spawn.xml';

Post automatically merged: Jun 14, 2020

Alright got it, you use $dataPath for the other 3 dirs as well so it had to be like this:

// path to folder with monster, npc and spawns data
$dataPath = 'C:\xampp\htdocs\map_viewer\data\\';
$monstersDir = 'monster\\';
$npcDir = 'npc\\';
$spawnsFile = 'world/Dolport-spawn.xml';

 

[Olddies]

this thing work flawless very usefull thanks for this!!

 

[Deaktiver]

1. is there any way to change the path of generated image files (otclient output)? Last time I've exported my map (97 MB otbm) I got 30GB of images. Don't have that much space available on my ssd. (I've splitted it into 5 parts; took ages for me .-. ).

2. do I have to copy the generted "map" folder to "website_and_php_files/" or "website_and_php_files/map_viewer"?

Thank you anyways for this great tool!

 

[Gesior.pl]

1. is there any way to change the path of generated image files (otclient output)? Last time I've exported my map (97 MB otbm) I got 30GB of images. Don't have that much space available on my ssd. (I've splitted it into 5 parts; took ages for me .-. ).

2. do I have to copy the generted "map" folder to "website_and_php_files/" or "website_and_php_files/map_viewer"?

Thank you anyways for this great tool!

1. It's not possible - in case of linux you can make some symlink / partition that will 'be otclient folder'.
2. You do not have to copy, just 'move' (not need space x2 for it).

 

[Deaktiver]

1. It's not possible - in case of linux you can make some symlink / partition that will 'be otclient folder'.
2. You do not have to copy, just 'move' (not need space x2 for it).

getting the "bad for loop variable" error when trying to run the script.

 

[Gesior.pl]

getting the "bad for loop variable" error when trying to run the script.

View attachment 46993

DashAsBinSh - Ubuntu Wiki (https://wiki.ubuntu.com/DashAsBinSh) :

In Ubuntu 6.10, the default system shell, /bin/sh, was changed to dash (the Debian Almquist Shell);

My script is 'bash' script. You can run it by:

/bin/bash linux_run_all.sh

or [it will auto detect its bash, not dash]:

./linux_run_all.sh

EDIT:
Using sh command to run .sh file looks like a good idea, but in case of Debian, it forces use of 'dash' in place of 'bash'.

 

[Deaktiver]

DashAsBinSh - Ubuntu Wiki (https://wiki.ubuntu.com/DashAsBinSh) :

My script is 'bash' script. You can run it by:

/bin/bash linux_run_all.sh

or [it will auto detect its bash, not dash]:

./linux_run_all.sh

working now.

thank you so much gesior <3

 

[Acubens]

Bro if i made changes on the map i need do all steps again right? theres not any cache to prevent convert it from beginning?

 

[Way20]

@Gesior.pl is it possible to scan just an specific area?

 

[Gesior.pl]

You can force .exe to generate .png only for some 'min X' to 'max X' area, but 'min X, min Y' to 'max X, max Y' limit is not possible.
Above these lines ( gesior/otclient_mapgen (https://github.com/gesior/otclient_mapgen/blob/master/otclientrc.lua#L145-L148) ):

    g_map.setMinXToLoad(currentMapPart.minXload)
    g_map.setMaxXToLoad(currentMapPart.maxXload)
    g_map.setMinXToRender(currentMapPart.minXrender)
    g_map.setMaxXToRender(currentMapPart.maxXrender)

Add these lines:

    local minX = 1500
    local maxX = 1700
    currentPart.minXrender = math.max(0, math.floor((minX - 8) / 8) * 8)
    currentPart.maxXrender = math.floor((maxX + 8) / 8) * 8
    currentPart.minXload = math.max(0, math.floor((currentPart.minXrender - 16) / 8) * 8)
    currentPart.maxXload = math.floor((currentPart.maxXrender + 16) / 8) * 8

ANYWAY it requires you to keep all original .png files from previous render to genereate zoom-out levels [with view of whole map], so I don't think it worth it. It's easier to generate whole map again.

 

[Way20]

You can force .exe to generate .png only for some 'min X' to 'max X' area, but 'min X, min Y' to 'max X, max Y' limit is not possible.

'min X' to 'max X' will do. Is it also possible to generate only for a specific floor?

ANYWAY it requires you to keep all original .png files from previous render to genereate zoom-out levels [with view of whole map], so I don't think it worth it. It's easier to generate whole map again.

I don't plan to use generated images for map view. I'm trying to find inconsistencies on my map using a python script to scan the difference between 2 images, I was previously using RME to take screenshots and compare but thats a bother, with your solution I could get images from both maps in a single day. This also leads me to another question, how can I modify it to take screenshots bigger than 8x8?

 

[Gesior.pl]

Is it also possible to generate only for a specific floor?

No. It genereates all floors and on every image are all floors below (with added shadow).

This also leads me to another question, how can I modify it to take screenshots bigger than 8x8?

There is available LUA function:

-- x, y, z - left-top corner of images/myfile
-- sizeInSQM - in case of generator it's '8', it can generate only rectangles
g_map.drawMap(filePath, x, y, z, sizeInSQM)

This should work to generate one big image:

prepareClient(1076, '/things/1076/items.otb', '/map.otbm')
g_map.loadOtbm('/map.otbm')
g_map.drawMap('images/myfile.png', 1000, 1000, 7, 50)

Remember to create folder 'images' in '%HOMEPATH%' before, it won't make it automatically - or use file name without folder.
Image size will be 50x50 tiles = 50*32*50*32 px = 2.560.000, 4 bytes per pixels.. 2560000 * 4 = 10.240.000, which is 10MB. You can generate much bigger images, but they are generated in RAM and you will need a lot of RAM.

Whole positions, ranges and all-floor rendering code is here:

otclient_mapgen/src/client/mapio.cpp at master · gesior/otclient_mapgen

Open Tibia map images generator [for LeafletJS]. Loads .otbm file and generate .png images 256x256 px - gesior/otclient_mapgen

github.com

You can easily adjust it to render any kind of image.

------------------------------
Generator could be fully managed by LUA and use only g_map.drawMap, but in case of 1.130.000 images of real map, it was 100x faster to manage queue of images in C++.

 

[Way20]

Add these lines:

local minX = 1500
local maxX = 1700
currentPart.minXrender = math.max(0, math.floor((minX - 8) / 8) * 8)
currentPart.maxXrender = math.floor((maxX + 8) / 8) * 8
currentPart.minXload = math.max(0, math.floor((currentPart.minXrender - 16) / 8) * 8)
currentPart.maxXload = math.floor((currentPart.maxXrender + 16) / 8) * 8

This didn't worked for me. At first it gave me an error:
View attachment 55248
then I adjusted variable name to same as on github's "currentPart > currentMapPart" and it generated the images but not for the conditions I defined which were: "minX = 33240 and maxX = 33250".

No. It genereates all floors and on every image are all floors below (with added shadow).

To change this to work on a single floor, source changes would be needed?

This should work to generate one big image:

I tried it, didn't work for me, am I doing something wrong?
View attachment 55251

You can generate much bigger images, but they are generated in RAM and you will need a lot of RAM.

That's not a problem for me, I have plenty of RAM

Generator could be fully managed by LUA and use only g_map.drawMap, but in case of 1.130.000 images of real map, it was 100x faster to manage queue of images in C++.

yes, I get it, for me I rather have one big image for an specific area, I wont be using the images to generate the web map.