Time for change on the map chewing memory?

[Printer]

Hello everyone,

Well, i just wonder will the map format change in the near future. Asfar i can see if we had full real tibia map and converted it to work on open-tibia. I can say the map will be 200-300mb big. That will chew alot of memory. Is there any backup plan tfs? Or shall we go with otbm 4ever?

Regrads,
Printer!

 

[fluidsonic]

Any suggestions for a simple yet fast, dense and maybe extensible format?

Maybe we could even split the map into separate files for different "areas" (most simple would be one for each floor).
That would allow partial updating and reloading at runtime

It could also make sense to further separate the plain map from the logic into different files (that is action IDs, unique IDs etc).

 

[Printer]

Well, i have no plan. But time for brain storming ideas ^^

 

[Evan]

If I'm not mistaken, CipSoft has their map set up in chunks, rather than one huge file.
This could be an option to consider, I'm not totally sure.

 

[Printer]

If I'm not mistaken, CipSoft has their map set up in chunks, rather than one huge file.
This could be an option to consider, I'm not totally sure.

Well, the only person which know is the one which have the real cipsoft files.

 

[J.Dre]

Loading parts of the map until the entire map is loaded might be better. I'm not sure if the usage would decrease, but I believe the entire format should be changed.

Edit: Memory is so cheap these days, it's not really a big deal.

 

[fluidsonic]

Yes, memory is cheap. But OT maps can easily explode in size.

Theoretical map maximum is around 60,000 x 60,000 x 16 which would be 481 GB in 64-bit pointers to Tile objects alone!

Okay assume we have a smaller map which averages at 2,000 x 2,000 x 14.
That is 56,000,000 tiles * 8-byte pointer = 448 MB.
sizeof(StaticTile) + sizeof(Item) is around 128 byte * 56m tiles = 7,168GB
Totaling at around 8GB minimum memory usage.

Actual usage is much higher because there is a lot more to set up and you cannot store the data efficiently. Esp. if the map has holes in it you cannot use just arrays to map positions to tile pointers.
And that's just the map - no monsters, no NPCs, no scripts, no dynamic items, etc.

So small sized maps = memory-only is fine.

Medium sized maps = loaded dynamically on-demand is fine unless there are too many players spread evenly across the entire map, which would load most of the map and you run OOM. and you also have to unload as soon as there is no player left in a chunk or your server will OOM over time anyway.

Large maps w/ many players = Nothing will help here. Either you have reeeeeally lots of memory (but I guess the server will still not run smoothly due to CPU usage) our you start clustering the game across multiple servers. But that's a whole different story I guess ^^

 

[Printer]

Yes, memory is cheap. But OT maps can easily explode in size.

Theoretical map maximum is around 60,000 x 60,000 x 16 which would be 481 GB in 64-bit pointers to Tile objects alone!

Okay assume we have a smaller map which averages at 2,000 x 2,000 x 14.
That is 56,000,000 tiles * 8-byte pointer = 448 MB.
sizeof(StaticTile) + sizeof(Item) is around 128 byte * 56m tiles = 7,168GB
Totaling at around 8GB minimum memory usage.

Actual usage is much higher because there is a lot more to set up and you cannot store the data efficiently. Esp. if the map has holes in it you cannot use just arrays to map positions to tile pointers.
And that's just the map - no monsters, no NPCs, no scripts, no dynamic items, etc.

So small sized maps = memory-only is fine.

Medium sized maps = loaded dynamically on-demand is fine unless there are too many players spread evenly across the entire map, which would load most of the map and you run OOM. and you also have to unload as soon as there is no player left in a chunk or your server will OOM over time anyway.

Large maps w/ many players = Nothing will help here. Either you have reeeeeally lots of memory (but I guess the server will still not run smoothly due to CPU usage) our you start clustering the game across multiple servers. But that's a whole different story I guess ^^

That sounds legit