Source's custom OT game engine (TypeScriptFTW)

[Source]

I've been working a little over a year or so on my own game engine from scratch written mainly in C/C++ possibly with some non-Lua scripting in there.
Thought I'd share a few random videos here for fun from 2021 before the year ends.
It runs natively on Windows, Linux, and probably MacOS as well without the need for MSVS or any other overly complicated or bloated setups, and uses objects.srv (ascii items attributes) instead of items.otb, monster.db (real tibia monster spawn file), sector map files (ascii real tibia map files format), etc, so there's no use of any binary data formats at all.
And it also has multicore support for map loading as you can see in the first video demo.

A short demo of running it on Windows (please excuse my mouse movement, I was on a laptop):

A for-fun aimpractice module I created in about an hour or so of time on server side:

Some recent work where I implemented a config.lua-like config with C speed access times and alteration of push stackpos behavior:

A long and boring video of progress on perfecting real tibia monster idle movement and spawning algorithm:

My patreon: Patreon

 

[Dip Set]

Bump

 

[Source]

Bump

Short and pretty minor server update.
I've spent most of my time working on OTClient fixes these past months, but worked on these TSS fixes in the last couple of days since I had to postpone the OTClient development for a few days:

• Fixed a few segfaults related to multiplayer stuff, more work needs to be done, but players can now sometimes / usually see one another walking and turning when on the same screen.
• Improved getStackPos() and added multiplayer sending of removing / transforming object with correct stackpos in handleItemsExpire() as shown in the video below.

Nothing particularly noteworthy, just thought I'd share since you bumped.

 

[Boy67]

Out of curiosity, considering TFS is already heavily developed, how come you decided to pursue this? What are your goals?

 

[LordCompi]

Looks good, good luck with project

 

[Source]

Looks good, good luck with project

Thank you.

Out of curiosity, considering TFS is already heavily developed, how come you decided to pursue this? What are your goals?

Some of the main goals I set out with this project was to:

1. Have fun.
2. Learn C/C++ and become a better coder overall.
3. Learn on the lowest level how OT engines works or can work internally without reading, studying and memorizing 83k lines of C++ code and another 90k lines of XML and Lua.

Eventually I'd like to create a unique OT server that can easily be hooked up with certain modern technologies, I want the engine to use plain text formats (which it does), so everything can be easily changed with a simple text editor and be simple and minimalist so it's easy to add completely new features or tweak behaviors easily such as my randomized monster movement which is only 10 lines of code and uses rand() to determine a direction then tries up to 4 directions to walk in and then sleeps.

Considering that the server is written mainly in C/C++ its performance shouldn't actually be much if at all worse than TFS.
The global monster walking for example will be hogging up a lot of CPU, but a traditional monster movement based on player proximity can be easily implemented and should in fact only be a couple of lines of code to add in.
Meanwhile if the monster movement gets annoying, for example when debugging player interaction bugs and inspecting I/O network data, I can simply stop all player movement with about 3 lines of code using /stop which simply sets a boolean that controls monster movement to true or false.

One more note:
I don't know much about TFS, but I heard recently about its function Map::getSpectators(...).
This is a function I use myself, but the code is much simpler and shorter and I think is a good example of how to write simpler code.
My equivalent function to this is gameGetPlayers(...) and is 7 lines of code:

if (!spectators.empty())
	spectators.clear();
for (auto player : vplayers) {
	if (width == 0 || height == 0 || (lots_of_math...))
		spectators.push_back(player);
}
return &spectators;

Compare that to TFS' code (note that it calls getSpectatorsInternal(...) at line 480 which is another 50 lines of code for a total of approx 130 lines of code).

These 7 lines of code is to me easier to read, easier to understand, easier to implement, and is faster or virtually the equivalent of TFS's performance since std::vector operations are incredibly fast, like <0.1µs for 1k players fast.

I've also benchmarked TFS vs TSS event scheduling performance:
Sequential execution

Parallel execution

These results shows that TSS' event scheduling is 3% slower than TFS' event scheduling when executed in sequence and is 8837% faster than TFS when events are ran in parallel (which I'll admit might be a fluke and have to do with my language's internal implementation of event execution resulting in all events being executed at once when the event delay is set to 0 or near-0), but the main point remains that a 3% difference is completely negligible.

I don't really have a specific goal set in stone for this server project, everyday I'm just happy that I haven't found some major design flaw of the server that requires any major rewrites, but eventually I hope to be launching an alpha test server sometime in the future for others to maybe enjoy and to test the server's stability (I think I've gotten rid of all major segfaults for now, but there's still more work to be done), and perhaps launch the server paired with a certain technology that I'll talk more about once implemented.

Overall I just enjoy working with it, adding stuff, fixing stuff, improving stuff, and getting one step closer to being able to run exactly the server I want in my preferred language, with my preferred features, where I know all the code, etc, and enjoying the challenge throughout.

TL;DR My goal with the project is just to work on it and see where it goes and have a lot of fun in the meantime.

 

[nasu]

My equivalent function to this is gameGetPlayers(...) and is 7 lines of code:

I'm glad you care about clean code. In my opinion functions should be small, I will even say something more: functions should be smaller than they are. Then it is better to understand/read the code by another programmer. I don't program in C++ at the moment, but once I was writing a game and I created a disgusting function that had over 50 lines of code. I broke that function into several smaller functions.
Good luck with your project!

ps. Do you have a github account?

 

[oen432]

One more note:
I don't know much about TFS, but I heard recently about its function Map::getSpectators(...).
This is a function I use myself, but the code is much simpler and shorter and I think is a good example of how to write simpler code.
My equivalent function to this is gameGetPlayers(...) and is 7 lines of code:

if (!spectators.empty())
spectators.clear();
for (auto player : vplayers) {
if (width == 0 || height == 0 || (lots_of_math...))
spectators.push_back(player);
}
return &spectators;

Compare that to TFS' code (note that it calls getSpectatorsInternal(...) at line 480 which is another 50 lines of code for a total of approx 130 lines of code).

But getSpectators is not only for players. It also works for monsters, npcs and is supporting multiple floors. Can your function do the same?

 

[Source]

I'm glad you care about clean code. In my opinion functions should be small, I will even say something more: functions should be smaller than they are. Then it is better to understand/read the code by another programmer. I don't program in C++ at the moment, but once I was writing a game and I created a disgusting function that had over 50 lines of code. I broke that function into several smaller functions.
Good luck with your project!

ps. Do you have a github account?

Yeah, I understand software gets more complex as it gets more sophisticated, but sophisticated software doesn't necessarily have to be overly complex, convoluted or bloated with few if any comments.
But to their credit the software seem pretty robust and good at what it does.
My github is pretty empty but can be found here.
Thank you.

But getSpectators is not only for players. It also works for monsters, npcs and is supporting multiple floors. Can your function do the same?

Yeah, I noticed it has a parameter onlyPlayers. I'm not sure what the purpose of the function really is if it's going to get non-players since only players has sockets and sockets are the only thing you have to operate on when stuff happens.

If I did want to implement this I could simply add another loop for monsters and one for npcs and/or use my general creature pointers vector vcreatures to get everything in one loop with anything between 0 to 8 added lines depending on your choice of implementation.

And yeah the function supports multi-floor, of course.

 

[oen432]

Yeah, I noticed it has a parameter onlyPlayers. I'm not sure what the purpose of the function really is if it's going to get non-players since only players has sockets and sockets are the only thing you have to operate on when stuff happens.

No...? Getting monsters around players for spells, quests or other scripts? NPCs for quests? Checking if area has any monsters/players?

Btw. less code != better code. For you it might be 50 lines, for the compiler it ends up being 10 and even less, based on the code.

 

[Boy67]

Thank you.


Some of the main goals I set out with this project was to:

1. Have fun.
2. Learn C/C++ and become a better coder overall.
3. Learn on the lowest level how OT engines works or can work internally without reading, studying and memorizing 83k lines of C++ code and another 90k lines of XML and Lua.

Eventually I'd like to create a unique OT server that can easily be hooked up with certain modern technologies, I want the engine to use plain text formats (which it does), so everything can be easily changed with a simple text editor and be simple and minimalist so it's easy to add completely new features or tweak behaviors easily such as my randomized monster movement which is only 10 lines of code and uses rand() to determine a direction then tries up to 4 directions to walk in and then sleeps.

Considering that the server is written mainly in C/C++ its performance shouldn't actually be much if at all worse than TFS.
The global monster walking for example will be hogging up a lot of CPU, but a traditional monster movement based on player proximity can be easily implemented and should in fact only be a couple of lines of code to add in.
Meanwhile if the monster movement gets annoying, for example when debugging player interaction bugs and inspecting I/O network data, I can simply stop all player movement with about 3 lines of code using /stop which simply sets a boolean that controls monster movement to true or false.

One more note:
I don't know much about TFS, but I heard recently about its function Map::getSpectators(...).
This is a function I use myself, but the code is much simpler and shorter and I think is a good example of how to write simpler code.
My equivalent function to this is gameGetPlayers(...) and is 7 lines of code:

if (!spectators.empty())
    spectators.clear();
for (auto player : vplayers) {
    if (width == 0 || height == 0 || (lots_of_math...))
        spectators.push_back(player);
}
return &spectators;

Compare that to TFS' code (note that it calls getSpectatorsInternal(...) at line 480 which is another 50 lines of code for a total of approx 130 lines of code).

These 7 lines of code is to me easier to read, easier to understand, easier to implement, and is faster or virtually the equivalent of TFS's performance since std::vector operations are incredibly fast, like <0.1µs for 1k players fast.

I've also benchmarked TFS vs TSS event scheduling performance:
Sequential execution

Parallel execution

These results shows that TSS' event scheduling is 3% slower than TFS' event scheduling when executed in sequence and is 8837% faster than TFS when events are ran in parallel (which I'll admit might be a fluke and have to do with my language's internal implementation of event execution resulting in all events being executed at once when the event delay is set to 0 or near-0), but the main point remains that a 3% difference is completely negligible.

I don't really have a specific goal set in stone for this server project, everyday I'm just happy that I haven't found some major design flaw of the server that requires any major rewrites, but eventually I hope to be launching an alpha test server sometime in the future for others to maybe enjoy and to test the server's stability (I think I've gotten rid of all major segfaults for now, but there's still more work to be done), and perhaps launch the server paired with a certain technology that I'll talk more about once implemented.

Overall I just enjoy working with it, adding stuff, fixing stuff, improving stuff, and getting one step closer to being able to run exactly the server I want in my preferred language, with my preferred features, where I know all the code, etc, and enjoying the challenge throughout.

TL;DR My goal with the project is just to work on it and see where it goes and have a lot of fun in the meantime.

Looks cool mate.

I too looked into a different technology and now have a very fast, robust and efficient web version of tibia. As you said, I am able to edit the client, server and map editor at any time, instantly, all written in the same language and sharing the same shared resources.

In regards to the monster walking, that's how every tibia like game does it. If there is no path, it chooses a random available tile to wander and the sleeps until the next think cycle.

Yeah, I noticed it has a parameter onlyPlayers. I'm not sure what the purpose of the function really is if it's going to get non-players since only players has sockets and sockets are the only thing you have to operate on when stuff happens.

When you start coding more of the server, you will find you will need to grab other entities such as npcs and monsters, how else are you meant to get data from the server to the client, of where monsters and npcs are on the server?

As oen432 says, sometimes less code doesn't always mean better code. For example, TFS' getSpectators function includes caching, which in turn will make executing a similar call extremely faster, even faster than your 7 lines of code as it doesn't need to iterate through any map or world array, and you are only searching for players.

But all in all, its a nice start. But I would suggest not benchmarking until you have a full system. You can of course benchmark things like your path-finding etc, but only on a comparable scale such as large map, obstacles, creatures etc.

 

[Source]

No...? Getting monsters around players for spells, quests or other scripts? NPCs for quests? Checking if area has any monsters/players?

Sure. What arguments would you provide to this function to get all the players on the screen of say a GFB or a diagonal Energy Wall short of calling it for every single individual tile?

Btw. less code != better code. For you it might be 50 lines, for the compiler it ends up being 10 and even less, based on the code.

I see. So if you were given the job to do one specific coding task, say write a function that accomplishes something, and you had the option to write either 5000 lines of code to accomplish the task or 1 you would choose the 5000 lines?

 

[oen432]

Sure. What arguments would you provide to this function to get all the players on the screen of say a GFB or a diagonal Energy Wall short of calling it for every single individual tile?

I don't quiet understand this question.

... Have you ever optimized anything in your life?

Wait, you really believe that less code is better? Less code is less functionality at most. Your code proves the case.

 

[Sarah Wesker]

I hope you continue with your project @Source you can see the excitement you have for it

I would like to delight my eyes a little with more code
And I also hope for more tests and comparisons that can be validated in some way

 

[Boy67]

Sure. What arguments would you provide to this function to get all the players on the screen of say a GFB or a diagonal Energy Wall short of calling it for every single individual tile?


I see. So if you were given the job to do one specific coding task, say write a function that accomplishes something, and you had the option to write either 5000 lines of code to accomplish the task or 1 you would choose the 5000 lines?

1. If you've just been teleported somewhere, how would you get the data of all entities including players, monsters and NPC's

2. That question is a bad one, if you can get the same result from say 50 lines vs 150 lines then yes, but just because your 7 lines of code is less than TFS's 150 lines of code, doesn't mean anything, the function TFS uses calculate a lot more stuff and includes caching....

 

[oen432]

I see. So if you were given the job to do one specific coding task, say write a function that accomplishes something, and you had the option to write either 5000 lines of code to accomplish the task or 1 you would choose the 5000 lines?

That made me chuckle, good one bro.

 

[Source]

Looks cool mate.

I too looked into a different technology and now have a very fast, robust and efficient web version of tibia. As you said, I am able to edit the client, server and map editor at any time, instantly, all written in the same language and sharing the same shared resources.

Cool.

In regards to the monster walking, that's how every tibia like game does it. If there is no path, it chooses a random available tile to wander and the sleeps until the next think cycle.

That's not true. First off what I said was that I'm using the equivalent of an addEvent for each monster upon spawn. This is the simplest option possible and makes all monsters across the entire map walk randomly which is not the norm at all.
Secondly not all engines has monsters walking in random directions when they have no target, for the longest time basically all OTS had no movement randomization at all. And thirdly most OTS that has movement randomization doesn't use the proper real tibia algorithm, monsters are usually sluggish, for instance walking two steps in a row then sleeping for several seconds (e.g. mastercores if my memory serves me right here) or (often) simply sleeps for too long.
The funny part is it took me no effort to implement at all. It's a super simple algorithm to implement, all it (probably) takes is a simple enough engine to let you write code simply.

When you start coding more of the server, you will find you will need to grab other entities such as npcs and monsters, how else are you meant to get data from the server to the client, of where monsters and npcs are on the server?

Monster movement (data sent to sockets) is sent upon monster movement, lol, that's how it's tracked.
The walk function sends gameMoveCreature(...) which calls gameGetPlayers(...) that gets all the surrounding players which simply iterates over a vector of player pointers and calculates if the position is within range or not.
It was trivial to implement a variable width/height set of tiles to adjust to customized clients with more or fewer tiles as well.

As oen432 says, sometimes less code doesn't always mean better code. For example, TFS' getSpectators function includes caching, which in turn will make executing a similar call extremely faster, even faster than your 7 lines of code as it doesn't need to iterate through any map or world array, and you are only searching for players.

I think you're relying heavily on assumptions here. I don't have the answer on whether my function is faster or not, nor do you, so neither of us should pretend to know the answer here unless you can say something specific about TFS implementation other than "it uses caching so it must be faster", you might as well say that MySQL is faster than vectors because "MySQL uses caching".
Considering that leafs are more complex than vectors, just like maps are, they're going to be more costly to operate on.
I'm open to having my mind changed here since leafs isn't something I know too much about, but "it uses caching" is not it.

But all in all, its a nice start. But I would suggest not benchmarking until you have a full system. You can of course benchmark things like your path-finding etc, but only on a comparable scale such as large map, obstacles, creatures etc.

Why?

That made me chuckle, good one bro.

 

[oen432]

Monster movement (data sent to sockets) is sent upon monster movement, lol, that's how it's tracked.

How player knows that there is a creature on the screen after teleport? What if you teleport and NPC/Monster doesn't move? Do you iterate through all the creatures on the same tile every time player teleports to X location? Caching helps because you can have 2 players using same spell that has getSpectators in the code and the center and range is the same. Instead of iterating twice and doing all the calculations, you can use cache.

 

[Source]

How player knows that there is a creature on the screen after teleport?

gameMoveCreature(...).

What if you teleport and NPC/Monster doesn't move? Do you iterate through all the creatures on the same tile every time player teleports to X location?

I'm not sure what the significance of the first part of your question is here. If another player/monster/NPC doesn't move then there's nothing to do or calculate. If a player/monster/NPC moves, it's tracked by gameCreatureMove(...).
Unlike TFS(?) I don't iterate over tiles, I iterate over players, if you do the math you can see why.

Caching helps because you can have 2 players using same spell that has getSpectators in the code and the center and range is the same. Instead of iterating twice and doing all the calculations, you can use cache.

Again, is MySQL faster than iterating over vectors since it uses caching?

 

[Boy67]

Monster movement (data sent to sockets) is sent upon monster movement, lol, that's how it's tracked.
The walk function sends gameMoveCreature(...) which calls gameGetPlayers(...) that gets all the surrounding players which simply iterates over a vector of player pointers and calculates if the position is within range or not.
It was trivial to implement a variable width/height set of tiles to adjust to customized clients with more or fewer tiles as well.

Scenario:

You've just teleported, and 3 players have trapped a demon in the corner.

How are you going to receive all the data about the demon, as it hasn't called gameMoveCreature()?

That's not true. First off what I said was that I'm using the equivalent of an addEvent for each monster upon spawn. This is the simplest option possible and makes all monsters across the entire map walk randomly which is not the norm at all.
Secondly not all engines has monsters walking in random directions when they have no target, for the longest time basically all OTS had no movement randomization at all.

Are you firing the event upon spawn? or upon waking up? Please don't tell me all you're monsters are constantly moving with no spectators. And you can just have random slightly different think/move intervals to make the creatures think independently of each other.

And for your second point, please tell me the engines that don't have random walking if no target, and what do they use if not, a pattern?