OpenTibia OTClient that unpack map.otbm to .png files - v4.0

[breispodeu]

Hello.

For some reason the zoom 14 is not loading parts of the map (all floors with zoom 14 in the same area) for me. Only the zoom 14 is where I found this issue.




But in the same area in all other zooms, it is ok:

 

[breispodeu]

Hello.

For some reason the zoom 14 is not loading parts of the map (all floors with zoom 14 in the same area) for me. Only the zoom 14 is where I found this issue.

May be related: I'm trying to generate my map but always there is the same part missing on the same zoom level.

The first thing I notice was this:



It says 121947 images generated, but there is only 120439 images in the folder... Tried to use different values for how many parts it should split, but I always ended with missing images for some parts on specific zoom levels (few images on zoom 14 and zoom 13, for all floors).

EDIT: As my understanding since the images generated from the OTClient will be the 16 zoom images, but there is no failure on this zoom level. So probably it is not related to my problem.

Then I read the code you made to generate the zoom files on PHP and the linux script. I think it may be race condition. You start all process to generate zoom images at the same time. Probably it will have few issues if there is multiple threads generating images (each zoom level depends on the previous zoom level images).

I'm trying too different fixes:

1 - Simply remove the concurrence and run level by level (removing & from the script)
2 - Modify the code to always use the base images from zoom 16 instead of the 4 images of the previous zoom

After testing it I will come back to say if it is fixed.

 

[Gesior.pl]

It says 121947 images generated, but there is only 120439 images in the folder

I think it's normal. Number showed in OTC is number of different 8x8 SQM squares it will try to generate, but if all tiles in that area have no items (black screen), it does not save image on HDD, as not existing images are also visible in web browser as black screen.
Map images generator detects 8x8 squares that have some items inside and only generates them (to skip millions of empty area tiles), but then it adds ~20 tiles around detected map area, to make sure that other floor items are loaded too (they are offset by +/- floor count SQM in client), so there are some 8x8 squares with no items on list of 'images to generate'.

As my understanding having less images in this first generation process should make all zooms to have missing images not only specific ones, right?

It's not related. Full 'zoom' images of 8x8 SQM squares are 256x256 pixels size and they are visible in web browser on zoom level 16 ( https://map.kasteria.online/#zoom,16,position,32352,31705,7 ).
17-18 zoom levels are 'zoom 16' images scaled up by web browser ( https://map.kasteria.online/#zoom,18,position,32356,31703,7 ).
0-15 zoom levels are generated by PHP scripts you run after map generation in OTC (ex. 14 zoom: https://map.kasteria.online/#zoom,14,position,32352,31730,7 ).

So if there are any images missing on floor 14, that are visible on floor 16, there must be a problem with PHP script that generates them.

Zoom level 15 images are made by merging 4 images from zoom level 16 and shrinking by 50% to generate 256x256 images.
Zoom level 14 images are made by merging 4 images from zoom level 15 and shrinking by 50% to generate 256x256 images.
So if image on zoom level 14 is bugged[black]/missing, all next zoom levels (13, 12, 11..) should be bugged too.

Maybe it's some problem with opening .htm file on local PC. You can try to debug it in web browser by clicking F12 and going Network tab, to view what images it loads. Can you find URLs to these 14 zoom level images that are missing (should be red in network tab)? Are they missing in map/14 folder or it's just web browser problem, that it can't load them?
ex. there is some black area 8x8 on map, so it returned 404 from server and it's red in Network tab, but it's fine in my case, because on zoom level 16 it's also black:

 

[breispodeu]

I think it's normal. Number showed in OTC is number of different 8x8 SQM squares it will try to generate, but if all tiles in that area have no items (black screen), it does not save image on HDD, as not existing images are also visible in web browser as black screen.
Map images generator detects 8x8 squares that have some items inside and only generates them (to skip millions of empty area tiles), but then it adds ~20 tiles around detected map area, to make sure that other floor items are loaded too (they are offset by +/- floor count SQM in client), so there are some 8x8 squares with no items on list of 'images to generate'.

It's not related. Full 'zoom' images of 8x8 SQM squares are 256x256 pixels size and they are visible in web browser on zoom level 16 ( https://map.kasteria.online/#zoom,16,position,32352,31705,7 ).
17-18 zoom levels are 'zoom 16' images scaled up by web browser ( https://map.kasteria.online/#zoom,18,position,32356,31703,7 ).
0-15 zoom levels are generated by PHP scripts you run after map generation in OTC (ex. 14 zoom: https://map.kasteria.online/#zoom,14,position,32352,31730,7 ).

So if there are any images missing on floor 14, that are visible on floor 16, there must be a problem with PHP script that generates them.

Zoom level 15 images are made by merging 4 images from zoom level 16 and shrinking by 50% to generate 256x256 images.
Zoom level 14 images are made by merging 4 images from zoom level 15 and shrinking by 50% to generate 256x256 images.
So if image on zoom level 14 is bugged[black]/missing, all next zoom levels (13, 12, 11..) should be bugged too.

Maybe it's some problem with opening .htm file on local PC. You can try to debug it in web browser by clicking F12 and going Network tab, to view what images it loads. Can you find URLs to these 14 zoom level images that are missing (should be red in network tab)? Are they missing in map/14 folder or it's just web browser problem, that it can't load them?
ex. there is some black area 8x8 on map, so it returned 404 from server and it's red in Network tab, but it's fine in my case, because on zoom level 16 it's also black:
View attachment 88762

I see..

But yeah, the files is missing in the folder, not a network problem. I'm running it locally first to test, so should be pretty hard to be a network problem.

EDIT:

You said that if the zoom level 14 is missing, probably all next zoom levels should be bugged too, right?

Thats not what is hapenning for me hahaha Check this out:

Zoom 14:


Zoom 13:



Zoom 15:


These are the result of generating the zoom level images one level by level, without running parallel scripts..

And it is not only in one area or one level. In the right side of my map there is an issue, but in the zoom 13:



And both zoom s14 and 12 are ok:



One thing both areas have in common is that all images from the same column (same X) are missing.. For example from 370,-220,7 to 370,1285,7 there is no images in this column. And all floors are missing images in these coordinates, but only for the specific zoom.

EDIT: I really think it is something with the PHP script to generate the zoom tiles, just don't know what yet.. Read the script and it seems fine to me. Because it is deterministic. Every single time I regenerate everything from scratch, the same areas had missing images.

 

[Gesior.pl]

But yeah, the files is missing in the folder, not a network problem. I'm running it locally first to test, so should be pretty hard to be a network problem.

IDK what to say. I've ran that code 30+ times on Linuxes and Windows with maps .otbm 10-190 MB. Nobody reported any problem.
I use Windows 10/Linux based on Ubuntu 20.04/22.04 and no antivirus. IDK what could stop writing to HDD/SSD.

Code that generates given floor/zoom image does not report any errors/warnings:

otclient_mapgen/website_and_php_files/2_tile_generator.php at master · gesior/otclient_mapgen

Open Tibia map images generator [for LeafletJS]. Loads .otbm file and generate .png images 256x256 px - gesior/otclient_mapgen

github.com

as it expects images to save on HDD/SSD. Often there are 1kk images to save (RL map) and breaking on single image error would require whole 'floor' generation to start again.

If you can share server items.otb and map .otbm and client .spr and .dat, message me on Discord: gesior.pl, I will try to fix any errors related to map generation.

EDIT:
If you can't share your server files, you can share level 16 zoom files and I can try to find resizing (zoom generation) bug, but it will take few GBs in .zip.

 

[breispodeu]

IDK what to say. I've ran that code 30+ times on Linuxes and Windows with maps .otbm 10-190 MB. Nobody reported any problem.
I use Windows 10/Linux based on Ubuntu 20.04/22.04 and no antivirus. IDK what could stop writing to HDD/SSD.

Code that generates given floor/zoom image does not report any errors/warnings:

otclient_mapgen/website_and_php_files/2_tile_generator.php at master · gesior/otclient_mapgen

Open Tibia map images generator [for LeafletJS]. Loads .otbm file and generate .png images 256x256 px - gesior/otclient_mapgen

github.com

as it expects images to save on HDD/SSD. Often there are 1kk images to save (RL map) and breaking on single image error would require whole 'floor' generation to start again.

If you can share server items.otb and map .otbm and client .spr and .dat, message me on Discord: gesior.pl, I will try to fix any errors related to map generation.

EDIT:
If you can't share your server files, you can share level 16 zoom files and I can try to find resizing (zoom generation) bug, but it will take few GBs in .zip.

I will send you the images, my internet upload this in 5 minutes haha Check discord for invitation from Agent..

Thanks

 

[leonardovsilva]

How to fix this?

 

[Gesior.pl]

How to fix this?

It looks like you are using "transparency" feature in .spr.
You have to enable it in otclient_mapgen.
Edit modules/game_things/things.lua and under:

local errorMessage = ''

add line:

g_game.enableFeature(GameSpritesAlphaChannel)

 

[Pifafa]

Is it possible to use this with the Nostaurius system?

 

[Renan1948]

What is new in version 4.0?

• website view with NPCs and monsters search
• configurable town names and positions - big names visible on map
• possibility to link NPCs and monsters ex. mymap.com/#npc,Xodet , mymap.com/#monster,Dragon+Lord
• scripts to run 'zoom levels' and 'compression' on 16 cores on windows (for linux it was in version 3.0)
• house images generator - generate images of houses that you can use on 'houses' page in acc. maker
• minimap generator - you can generate full minimap and use it with any OTClient
• sources updated for new linuxes (new openssl) and newest Visual Studio project from edubart/otclient (vcpkg)

Github release with 64-bit .exe:

Release Map generator v4 - Windows x64 · gesior/otclient_mapgen

Open Tibia map images generator [for LeafletJS]. Loads .otbm file and generate .png images 256x256 px - Release Map generator v4 - Windows x64 · gesior/otclient_mapgen

github.com

What is this?
It's automatic map image generator [for website view 'like GoogleMaps'].
It generates OTS map .png images (each 8x8 tiles).

-- BINARY (.EXE) ATTACHED! YOU DO NOT HAVE TO COMPILE! --
To run binary you may need to install Microsoft VS 2019 Redist x64 first:

https://aka.ms/vs/16/release/vc_redist.x64.exe

It contains configured JavaScript and HTML to view map on website like this:
View attachment 46550

How to use?
1. Download otclient_mapgen from GitHub:
GitHub - gesior/otclient_mapgen: Open Tibia map images generator [good for Leaflets]. Loads .otbm file and generate .png images 256x256 px

and compile it like normal OTClient:
Compiling on Windows · edubart/otclient Wiki · GitHub
Compiling on Linux · edubart/otclient Wiki · GitHub
Compiling on Mac OS X · edubart/otclient Wiki · GitHub
Compiling for Android · edubart/otclient Wiki · GitHub

OR DOWNLOAD BINARY (.EXE) ATTACHED

2. Copy your server client .spr and .dat files to OTClient folder: data/things/HERE_PROTOCOL_VERSION/

3. Copy your server data/items/items.otb file to OTClient folder: data/things/HERE_PROTOCOL_VERSION/

4. Copy your server data/world/MAP_NAME.otbm file to OTClient folder: data/map.otbm

5. Run otclient.exe

6. Type in client terminal command like:

prepareClient(1076, '/things/1076/items.otb', '/map.otbm', 8, 5)

• with you client protocol version,
• valid paths to items.otb and map.otbm,
• number of threads to run (in example it's 8),
• number of map parts (OTC will split map to parts to reduce maximum RAM usage)

[Paths are relational, so start path with '/' to start in folder 'data', access to other folders in computer is blocked]

7. OTClient will show in 10-50 seconds (it will freez client, do not close it, just wait) something like:


8. Type in client terminal command:

generateMap('all', 30)

OTClient will report progress in terminal.

9. Your map images will appear in your system 'user' directory

%HOMEPATH%/otclient - Windows, open it in folder explorer
$({HOME}/.otclient - Linux, 'cd' to it, this folder is invisible, but you can navigate to it

10. Copy folder map from folder metioned in previous step to folder website_and_php_files of otclient_mapgen.

--- NEXT STEPS REQUIRE PHP INSTALLED IN SYSTEM ---

--- LINUX: FOR GENERATION TIME, SET FOLDER website_and_php_files RIGHTS TO 777 ---

11. Linux

1. Execute (in terminal):
   sh linux_run_all.sh

11. Windows
1. If you got Git BASH installed

1. Execute (in system terminal):
   ./windows_run_1_pre_tile_generator.sh
   
   
2. Execute:
   ./windows_run_2_tile_generator.sh
   
   Watch CPU usage (by php.exe). Wait until it finish.
   
   
3. Execute:
   ./windows_run_3_pre_compress.sh
   
   
4. Execute:
   ./windows_run_4_compress.sh
   
   Watch CPU usage (by php.exe). Wait until it finish.

2. If you do not have Git BASH

1. Execute (in system terminal):
   php 1_pre_tile_generator.php
   
   
2. Execute (command parameter is map 'floor'):
   • You can execute these commands in any order.
   • You can open few terminals and run few commands at once to generate it faster (use all CPU cores).
     
     php 2_tile_generator.php 0
     php 2_tile_generator.php 1
     php 2_tile_generator.php 2
     php 2_tile_generator.php 3
     php 2_tile_generator.php 4
     php 2_tile_generator.php 5
     php 2_tile_generator.php 6
     php 2_tile_generator.php 7
     php 2_tile_generator.php 8
     php 2_tile_generator.php 9
     php 2_tile_generator.php 10
     php 2_tile_generator.php 11
     php 2_tile_generator.php 12
     php 2_tile_generator.php 13
     php 2_tile_generator.php 14
     php 2_tile_generator.php 15
     php 2_tile_generator.php 16
     
3. Execute:
   php 3_pre_compress.php
   
   
4. Execute (command parameter is map 'floor'):
   • You can set compression quality in file 4_compress.php (line: $quality = 80
   • You can execute these commands in any order.
   • You can open few terminals and run few commands at once to generate it faster (use all CPU cores).
     
     php 4_compress.php 0
     php 4_compress.php 1
     php 4_compress.php 2
     
     
     
     
     
     
     
     
     
     
     
     
     

12. Move folder website_and_php_files/map_viewer to your webserver - it contains all images and website scripts to view them (Leaflet Map).

13. Configure map_viewer. Config is at start of map.js file:
- You can keep images on other webserver. Set imagesUrl to URL of this server like: 'http://myserver.com/map_images/'

imagesUrl: 'map/', // URL to folder with 'zoom levels' (folders with names 0-16)
imagesExtension: '.jpg',
mapName: 'RL MAP?',
startPosition: {x: 1000, y: 1000, z: 7},
startZoom: 14,
minZoom: 4,
maxZoom: 18, // maximum zoom with full quality is 16

GENERATE HOUSE IMAGES

1. Do steps 1-6 from How to use? instruction (above).
2. Type in client terminal command:
   generateHouses(30, false)
   • 30 - shadow percent for tiles around house (they will be darker)
   • false - load whole map at once, requires much RAM, set it to true to load map by parts, it's much slower, but will use little amount of RAM to generate house images
3. It will freez client for house generation time! Don't close it. If you want watch progress run OTClient by windows/linux console (not by clicking .exe):
   ./otclient
   'Freezed' client will show messages about generated houses in system console.
4. Your house images will appear in your system 'user' directory:
   %HOMEPATH%/otclient/house/ - Windows, open it in folder explorer
   ${HOME}/.otclient/house/ - Linux, 'cd' to it, this folder is invisible, but you can navigate to it

Houses with few floors generate few images with 'floor level' in file name, but all floors images have same size and top-left corner position, so you can show it on website with +/- button to switch floors.
Example house image - tiles outside house are 30% darker:
View attachment 46551

GENERATE FULL MINIMAP FOR OTCLIENT

Most of released OTClient .exe files are 32-bit and they got limit around 2 GB RAM. It means that you cannot generate minimap from .otbm bigger than ~60MB.
Attached binary is 64-bit version of OTClient. It can load any size .otbm into RAM. To export full minimap, you got to load whole map.

1. Do steps 1-6 from How to use? instruction (above).
2. Type in client terminal command:
   saveMinimap('minimap.otmm')
3. Your minimap will appear in your system 'user' directory:
   %HOMEPATH%/otclient/ - Windows, open it in folder explorer
   ${HOME}/.otclient/ - Linux, 'cd' to it, this folder is invisible, but you can navigate to it
4. Copy it to folder data in your normal OTClient. It must be closed when you place there new minimap!
5. Run your OTClient, login to game, you should see full minimap.

How can I export a single image of the entire map? My map is very small, so I don't think there will be any problems, plus I have plenty of RAM.

 

[Gesior.pl]

Is it possible to use this with the Nostaurius system?

Probably not. This client requires items.otb, so you would need to somehow generate items.otb with item attributes and IDs as in Nostalrius server.

How can I export a single image of the entire map? My map is very small, so I don't think there will be any problems, plus I have plenty of RAM.

500x500 SQM (16k px x 16k px) takes around 1 GB RAM and generates ~60 MB PNG.

I've described it here:

Post in thread 'Export RME map'

Dec 19, 2024

How to export RME map to image: answer from private conversation with @leonardovsilva

You must use https://github.com/gesior/otclient_mapgen/releases/download/v5.1/otclient_mapgen_v5.1_win-x64.zip
Example of big image with TFS 1.4 files and Tibia 10.98 files put into data/things/1098/:

prepareClient(1098, '/things/1098/items.otb', '/things/1098/forgotten.otbm', 1, 1)

wait until it load.
Then (you can paste all these lines as 1 line into OTClient terminal, it will work):

g_map.setMinXToLoad(0)
g_map.setMaxXToLoad(70000)
g_map.setMinXToRender(0)...

• Gesior.pl

• 

More detailed instruction for TFS 1.4 protocol 10.98:
1. First download 5.1 version of generator, it has new drawCustomMap function:

https://github.com/gesior/otclient_mapgen/releases/download/v5.1/otclient_mapgen_v5.1_win-x64.zip

2. Then download your client .spr and .dat (or use version from your client, if you have modified files):

OTSME DL - data/tibia-clients/dat_and_spr

ex. https://downloads.ots.me/data/tibia-clients/dat_and_spr/1098.zip and unpack it into data/things in client, so there will be folder data/things/1098 with Tibia.dat and Tibia.spr
3. From OTS data/items/ copy items.otb to otclient data/items.otb
4. From OTS data/world copy .otbm file to otclient data ex. data/forgottenserver.otbm
5. Run otclient.exe
6. Type (you can copy and paste it all as 1 line into OTC terminal, it will work):

g_map.setShadowPercent(30)

prepareClient(1098, 'items.otb', 'forgotten.otbm', 1, 1)
g_map.setMinXToLoad(0)
g_map.setMaxXToLoad(70000)
g_map.setMinXToRender(0)
g_map.setMaxXToRender(70000)
g_map.loadOtbm('forgotten.otbm')

to prepare client.
7. Type:

g_map.drawCustomMap("100x100.png", 100, 350, 7, 200, 450, true);

to generate 100 x 100 image starting on position x = 100, y = 350, z =7 and ending on position x = 200, y = 450, z =7 (it always generate only 1 floor).
Image will be saved in OTC folder next to .exe.
Last parameter true is "render lower floors", when you render other floor than 7 (ground level). Lower floors are rendered a bit darker, it's set by g_map.setShadowPercent(30) to make lower floors 30% darker.

If your client uses some custom features ex. 'extended sprites' on 8.6 protocol, you must enable them in modules/game_things/things.lua file. More information about it:

Solved Post in thread 'Generating minimap from otclient error'

Dec 9, 2024

Im using 8.6 extended client

If you are using my 5.1 otclient_mapgen .exe ( https://github.com/gesior/otclient_mapgen/releases/download/v5.1/otclient_mapgen_v5.1_win-x64.zip ), it does not auto-detect 'extended sprites' .dat/.spr files for any protocol as OTCv8 does.
You got to edit modules/game_things/things.lua and under:

local errorMessage = ''

add line:

g_game.enableFeature(GameSpritesU32)

to enable that 9.60 feature (over 65k sprites) on 8.60 protocol.. and any other OTC custom features you use (ex. dat/spr editor "transparency" =...

• Gesior.pl

• 
• 

 

[Renan1948]

Provavelmente não. Este cliente requer items.otb, então você precisaria gerar de alguma forma items.otbcom atributos e IDs de itens como no servidor Nostalrius.

500x500 m² (16k px x 16k px) ocupam cerca de 1 GB de RAM e geram ~60 MB de PNG.

Eu descrevi isto aqui:
[CONTEÚDO INCORPORADO="post-2765061"] Exportar mapa RME (https://otland.net/threads/export-rme-map.290672/#post-2765061[/INCORPORADO])

Instruções mais fornecidas para o protocolo TFS 1.4 10.98:
1. Primeiro baixe a versão 5.1 do gerador, ela tem novas drawCustomMapfunções:

https://github.com/gesior/otclient_mapgen/releases/download/v5.1/otclient_mapgen_v5.1_win-x64.zip

2. Em seguida, baixe os arquivos .spr e .dat do seu cliente (ou use a versão do seu cliente, se tiver modificado os arquivos):

OTSME DL - data/tibia-clients/dat_and_spr

ex. https://downloads.ots.me/data/tibia-clients/dat_and_spr/1098.zipe descompacte-o data/thingsno cliente, então haverá uma pasta data/things/1098com Tibia.dateTibia.spr
3.data/items/ Da cópia do OTS items.otbpara o otclientdata/items.otb
4. Faça uma cópia OTS data/worlddo arquivo .otbm para otclient dataex.data/forgottenserver.otbm
5. Execute otclient.exe
6. Digite (você pode copiar e colar tudo em uma única linha no terminal OTC, funcionará):

g_map.setShadowPercent(30)

prepareClient(1098, 'itens.otb', 'esquecido.otbm', 1, 1)
g_map.setMinXToLoad(0)
g_map.setMaxXToLoad(70000)
g_map.setMinXToRender(0)
g_map.setMaxXToRender(70000)
g_map.loadOtbm('esquecido.otbm')

para preparar o cliente.
7. Tipo:

g_map.drawCustomMap("100x100.png", 100, 350, 7, 200, 450, verdadeiro);

para gerar uma imagem de 100 x 100 começando na posição x = 100, y = 350, z =7e terminando na posição x = 200, y = 450, z =7(sempre gera apenas 1 andar).
A imagem será salva na pasta OTC ao lado de .exe.
O último parâmetro trueé "renderizar andares inferiores", quando você renderiza um andar diferente do 7 (nível superior). Os andares inferiores são renderizados um pouco mais escuros. A configuração é g_map.setShadowPercent(30)para tornar os andares inferiores 30% mais escuros.

Se seu cliente utilizar alguns recursos personalizados, como "sprites estendidos" no protocolo 8.6, você deverá habilitá-los no modules/game_things/things.luaarquivo. Mais informações sobre isso:
[CONTEÚDO INCORPORADO="post-2764598"] Resolvido - Gerando minimapa a partir de erro otclient (https://otland.net/threads/generating-minimap-from-otclient-error.290595/#post-2764598[/INCORPORADO])

I tried using the command "g_map.drawCustomMap("100x100.png", 100, 350, 7, 200, 450, true);" but nothing happened. Sorry for the delay.

EDIT: I got it, I just wrote it like this
g_map.setShadowPercent(30)
prepareClient(1098, 'items.otb', 'forgotten.otbm', 1, 1)
g_map.setMinXToLoad(0)
g_map.setMaxXToLoad(70000)
g_map.setMinXToRender(0)
g_map.setMaxXToRender(70000)
g_map.loadOtbm('forgotten.otbm')
g_map.drawCustomMap("floor_7.png", 1000, 1000, 7, 1100, 1100, true)