• There is NO official Otland's Discord server and NO official Otland's server list. The Otland's Staff does not manage any Discord server or server list. Moderators or administrator of any Discord server or server lists have NO connection to the Otland's Staff. Do not get scammed!

TFS 1.X+ Optimize RAM

Fortera Global

Intermediate OT User
Joined
Nov 20, 2015
Messages
1,180
Solutions
2
Reaction score
117
What can I do to optimize the use of RAM? Script examples that can take up too much RAM?

tfs 1.x+

thanks
 
Solution
In 2018:
Get better dedic.

Script that consume too much RAM? Scripts with memory leak.
In many cases it's better to leave small memory leak in LUA to reduce CPU usage.

Example:
Some temporary table with NPC state.
PHP:
local function creatureSayCallback(cid, type, msg)
   if not npcHandler:isFocused(cid) then
      return false
   end
   local player = Player(cid)
   if isInArray({"soft", "boots", "repair", "soft boots"}, msg) then
      npcHandler:say("Do you want to repair your worn soft boots for 10000 gold coins?", cid)
      npcHandler.topic[cid] = 1
   elseif msgcontains(msg, 'yes') and npcHandler.topic[cid] == 1 then
...
There is table 'npcHandler.topic' indexed by 'cid'. 'cid' changes every relog, so after player...
In 2018:
Get better dedic.

Script that consume too much RAM? Scripts with memory leak.
In many cases it's better to leave small memory leak in LUA to reduce CPU usage.

Example:
Some temporary table with NPC state.
PHP:
local function creatureSayCallback(cid, type, msg)
   if not npcHandler:isFocused(cid) then
      return false
   end
   local player = Player(cid)
   if isInArray({"soft", "boots", "repair", "soft boots"}, msg) then
      npcHandler:say("Do you want to repair your worn soft boots for 10000 gold coins?", cid)
      npcHandler.topic[cid] = 1
   elseif msgcontains(msg, 'yes') and npcHandler.topic[cid] == 1 then
...
There is table 'npcHandler.topic' indexed by 'cid'. 'cid' changes every relog, so after player logout there stays something like:
PHP:
npcHandler.topic[12343234] = 1
There are 2 ways to fix it:

1. In onDisappear event of NPC remove key from table.
Change:
PHP:
function onCreatureDisappear(cid)
    npcHandler:onCreatureDisappear(cid)
end
To:
PHP:
function onCreatureDisappear(cid)
    npcHandler:onCreatureDisappear(cid)
    npcHandler.topic[cid] = nil // = nil deletes value from LUA table
end

2. Use GUID as table index, not 'cid'. It does not change when player relog, but it will stay in RAM after logout anyway. It will reduce RAM usage to 1 key=value pair per player:
PHP:
local function creatureSayCallback(cid, type, msg)
   if not npcHandler:isFocused(cid) then
      return false
   end
   local player = Player(cid)
   if isInArray({"soft", "boots", "repair", "soft boots"}, msg) then
      npcHandler:say("Do you want to repair your worn soft boots for 10000 gold coins?", cid)
      npcHandler.topic[player:getGuid()] = 1
   elseif msgcontains(msg, 'yes') and npcHandler.topic[player:getGuid()] == 1 then
...

Both fixes require some programmer who would analyse every NPC and add changes in proper places. Programmers are expensive, hardware is cheap.
Both fixes will increase CPU usage (call to :getGuid() or remove table value) and reduce RAM usage by few MB per day.

Can you buy faster CPU? Fastest you can get in OVH is 4.7GHz. Soon there will be 5.0GHz, but you can't pay extra 100 euro and get 9.5GHz.
Can you buy more RAM? Of course. 4.7GHz dedic has 64GB RAM. Server will small memory leaks after 24 hours online with 1000 online will use less then 8GB.
In case of serious applications that must run 24/7 without restarts - memory leaks are not acceptable. In case of OTS - it's not a big problem.
 
Solution
In this example you aren't showcasing memory leak but application level cache. *just saying*
 
In 2018:
Get better dedic.

Script that consume too much RAM? Scripts with memory leak.
In many cases it's better to leave small memory leak in LUA to reduce CPU usage.

Example:
Some temporary table with NPC state.
PHP:
local function creatureSayCallback(cid, type, msg)
   if not npcHandler:isFocused(cid) then
      return false
   end
   local player = Player(cid)
   if isInArray({"soft", "boots", "repair", "soft boots"}, msg) then
      npcHandler:say("Do you want to repair your worn soft boots for 10000 gold coins?", cid)
      npcHandler.topic[cid] = 1
   elseif msgcontains(msg, 'yes') and npcHandler.topic[cid] == 1 then
...
There is table 'npcHandler.topic' indexed by 'cid'. 'cid' changes every relog, so after player logout there stays something like:
PHP:
npcHandler.topic[12343234] = 1
There are 2 ways to fix it:

1. In onDisappear event of NPC remove key from table.
Change:
PHP:
function onCreatureDisappear(cid)
    npcHandler:onCreatureDisappear(cid)
end
To:
PHP:
function onCreatureDisappear(cid)
    npcHandler:onCreatureDisappear(cid)
    npcHandler.topic[cid] = nil // = nil deletes value from LUA table
end

2. Use GUID as table index, not 'cid'. It does not change when player relog, but it will stay in RAM after logout anyway. It will reduce RAM usage to 1 key=value pair per player:
PHP:
local function creatureSayCallback(cid, type, msg)
   if not npcHandler:isFocused(cid) then
      return false
   end
   local player = Player(cid)
   if isInArray({"soft", "boots", "repair", "soft boots"}, msg) then
      npcHandler:say("Do you want to repair your worn soft boots for 10000 gold coins?", cid)
      npcHandler.topic[player:getGuid()] = 1
   elseif msgcontains(msg, 'yes') and npcHandler.topic[player:getGuid()] == 1 then
...

Both fixes require some programmer who would analyse every NPC and add changes in proper places. Programmers are expensive, hardware is cheap.
Both fixes will increase CPU usage (call to :getGuid() or remove table value) and reduce RAM usage by few MB per day.

Can you buy faster CPU? Fastest you can get in OVH is 4.7GHz. Soon there will be 5.0GHz, but you can't pay extra 100 euro and get 9.5GHz.
Can you buy more RAM? Of course. 4.7GHz dedic has 64GB RAM. Server will small memory leaks after 24 hours online with 1000 online will use less then 8GB.
In case of serious applications that must run 24/7 without restarts - memory leaks are not acceptable. In case of OTS - it's not a big problem.


o_O Well, thank you I will mark your answer as the best later, but I will keep the topic so that more people can give more ideas, very good indeed!

PS. Do you advise making the both modifications or choosing only 1 of them? The onCreatureDisappear part seems to be easier and more practical, but if both can be applied together I'll try apply
 
Last edited by a moderator:
In this example you aren't showcasing memory leak but application level cache. *just saying*
After player logout, data connected with player is not removed from memory = memory leaks.
We got table that only grows and some parts of it won't be accessed anymore by program (every relog player gets new unique 'cid').

It's not 'c++ style' memory leak. Where you lose pointer to some object and it's memory is not accessible anymore.
It's LUA, script language with garbage collector, so by definition, it should be not possible to create object that is not accessible from anywhere.

I wouldn't say it's cache. It's table with information needed to execute script. You can't store it any other way. Only problem with this script is that, it does not remove table records after last possible use of given key.

@eldera
If you got some better example. I'm pretty sure a lot of people would like to see it and learn more about LUA optimization problems. That's what we do on forum :cool:
 
In this example you aren't showcasing memory leak but application level cache. *just saying*
As gesior say it's just simple variable that you store in table and it reside in the ram this is because this table is linked with npc so until you remove this npc the garbage collector will not get rid of the data inside the table.

In the example gesior provide the data save as creatureId and with how cipsoft designed their engine there should be limit of 805306368 unique ids of every players, monsters, npcs(TFS actually probably don't have that limit) so the whole table can get as big as 6442450944 bytes which is 6gb for one npc!(it will take a lot of time to get to that big table but it still possible)
There's also to say that fetching data witch brackets from table still iterate through some of the keys so it hurts cpu too but probably not as much as ram(depends of how tables are implemented in Lua).
 
Is there more examples guys? thank you so much @Gesior.pl I'll apply this "npcHandler.topic[cid] = nil // = nil deletes value from LUA table" to all npcs and will check later if changes
 
Is there more examples guys? thank you so much @Gesior.pl I'll apply this "npcHandler.topic[cid] = nil // = nil deletes value from LUA table" to all npcs and will check later if changes
you dont need to add on all of them. just the ones that use table values indexed as player id
 
you dont need to add on all of them. just the ones that use table values indexed as player id

What you mean? I dont need add in these all npcs?

nndlTaD.png


all npcs I have here works like this:
hastebin

so I need change all? looking this example
 
What you mean? I dont need add in these all npcs?

nndlTaD.png


all npcs I have here works like this:
hastebin

so I need change all? looking this example
everytime you use topic with uid as index you should do what my handsome boy @Gesior.pl says.
also using guid as index is better too
 

PHP:
function onCreatureDisappear(cid)
    npcHandler:onCreatureDisappear(cid)
    npcHandler.topic[cid] = nil // = nil deletes value from LUA table
end

also worth mentioning that the line

npcHandler.topic[cid] = nil

actually doesn't "delete" the value from the Lua table until the garbage collector is ran
 
After player logout, data connected with player is not removed from memory = memory leaks.
We got table that only grows and some parts of it won't be accessed anymore by program (every relog player gets new unique 'cid').

It's not 'c++ style' memory leak. Where you lose pointer to some object and it's memory is not accessible anymore.
It's LUA, script language with garbage collector, so by definition, it should be not possible to create object that is not accessible from anywhere.

I wouldn't say it's cache. It's table with information needed to execute script. You can't store it any other way. Only problem with this script is that, it does not remove table records after last possible use of given key.

@eldera
If you got some better example. I'm pretty sure a lot of people would like to see it and learn more about LUA optimization problems. That's what we do on forum :cool:

Actually before writing my post I only briefly looked at the example you provided and misunderstood some things.

I thought you were doing:

PHP:
npcHandler.topic[cid] = someExpensiveWork();

and the reason of not deleting it from memory was that it wouldn't have to be recalculated upon next user login (saving CPU by caching function results) . In this case it would be what I called "application level cache".

So I take back my previous statement :)
 
more than likely you aren't going to fix your memory leaks by optimizing tiny parts of code such as a few table indexes or unneeded variables
if you're really having that big of an issue with ram and truly believe it's from lua you can set the garbage collector to run faster and more aggressively by doing:
Lua:
collectgarbage("setpause", 100)
and maybe setting "setstepmul" to higher than 200
the likely issue at hand is you have a memory leak in your sources, which would be a lot bigger in terms of memory usage than a leak in lua
if you're running linux on your server you can use valgrind to detect memory leaks in your sources
 
Last edited:
more than likely you aren't going to fix your memory leaks by optimizing tiny parts of code such as a few table indexes or unneeded variables
if you're really having that big of an issue with ram and truly believe it's from lua you can set the garbage collector to run faster and more aggressively by doing:
Lua:
collectgarbage("setstepmul", 100)
collectgarbage("setpause", 100)
the likely issue at hand is you have a memory leak in your sources, which would be a lot bigger in terms of memory usage than a leak in lua
if you're running linux on your server you can use valgrind to detect memory leaks in your sources
What are the defaults on thoose values?
 
What are the defaults on thoose values?
i was wrong with setstepmul, lower values makes it slower
but theyre both defaulted to 200
setpause: each value after 100 is 1% more memory until garbage collector is ran again
so by default it's 200, meaning it runs a new cycle when the total memory is doubled
100 meaning it doesn't wait to run a new cycle
setstepmul defaulted to 200, runs the collector twice as fast as memory allocation
 
It actually don't change anything at all since it's a table inside the npchandler which is handled by npcsystem when release focus "self.topic[focus] = nil".
It was only example of how Lua can leak memory but it's only useful in npcs that uses external tables and don't "free" the variables.
 
I'm trying here to see how much LUA is consuming the RAM here, but its returning in Kbytes (I think), how to return in MB? Can someone help?

Code:
20:43 Memory Using: 36848.040039062.

Lua:
local textMem = collectgarbage("count")
    player:sendTextMessage(MESSAGE_STATUS_CONSOLE_BLUE, "Memory Using: " .. textMem .. ".")

I have 8GB in my VPS.
its showing here:
3HsDvs2.png


my screen show in htop:
6PVWYOL.png


Its seems my ram is consumed by C++ then? :O o_O
 
Last edited by a moderator:
just do collectgarbage("count") / 1024 for MB
and yes, if your process is using ~4.4 gigs of memory it's your sources, not Lua
have you restarted your server and watched memory as time goes on?
generally if you have a memory leak over time you'll see it keep growing and growing
 
Back
Top