• There is NO official Otland's Discord server and NO official Otland's server list. The Otland's Staff does not manage any Discord server or server list. Moderators or administrator of any Discord server or server lists have NO connection to the Otland's Staff. Do not get scammed!

[TUTORIAL] How to Lazily Optimize saving of player_storage

Flatlander

Species Developer
Premium User
Joined
Feb 17, 2009
Messages
2,459
Solutions
3
Reaction score
1,342
Location
Texas
Hi everyone!

It's been a long time since I've made a tutorial but I have been messing around on some OT Servers and noticed Server Saves sometimes taking a very long time.

If this is happening to you! It's usually due to 1 or 2 things.
Option 1: Saving house items
Option 2: Saving storage values.

If you have a billion storage values on your players that you use for everything, and you have a lot of players online, this can definitely slow down your server saves.
This is because every single time you save your server, it deletes all storage values for all players that are currently online, and then re-saves everything.

This means, if you have 2 million unique storage values, but only 3 of them changed since the player logged in, instead of updating just those 3, TFS deletes all 2 million storage values and saves them again.

I have a quick (lazy) fix that resolves this problem and might make your server saves super-fast!

So here is my tutorial on how to do this in TFS 1.X


First thing first.

PLEASE BACK UP YOUR SQL DATABASE BEFORE DOING ANYTHING NEW TO IT!! YOU SHOULD ALWAYS BACKUP EVERY SINGLE TIME YOU DO AN UPDATE THAT MIGHT EFFECT THE DATABASE!! I AM NOT RESPONSIBLE FOR THIS BREAKING YOUR LIVE SERVER BECAUSE YOU DID NOT WANT TO BE SAFE AND MAKE A QUICK BACKUP OF YOUR SQL DATABASE!!

Step1:
in player.h add:
C++:
std::map<uint32_t, int32_t> changedStorageMap;
It should look like this:

C++:
        std::map<uint8_t, OpenContainer> openContainers;
        std::map<uint32_t, DepotLocker_ptr> depotLockerMap;
        std::map<uint32_t, DepotChest*> depotChests;
        std::map<uint32_t, int32_t> changedStorageMap;
        std::map<uint32_t, int32_t> storageMap;

Then in player.cpp change:
C++:
void Player::addStorageValue(const uint32_t key, const int32_t value, const bool isLogin/* = false*/)
{
    if (IS_IN_KEYRANGE(key, RESERVED_RANGE)) {
        if (IS_IN_KEYRANGE(key, OUTFITS_RANGE)) {
            outfits.emplace_back(
                value >> 16,
                value & 0xFF
            );
            return;
        } else if (IS_IN_KEYRANGE(key, MOUNTS_RANGE)) {
            // do nothing
        } else {
            std::cout << "Warning: unknown reserved key: " << key << " player: " << getName() << std::endl;
            return;
        }
    }

    if (value != -1) {
        int32_t oldValue;
        getStorageValue(key, oldValue);

        storageMap[key] = value;

        if (!isLogin) {
            auto currentFrameTime = g_dispatcher.getDispatcherCycle();
            if (lastQuestlogUpdate != currentFrameTime && g_game.quests.isQuestStorage(key, value, oldValue)) {
                lastQuestlogUpdate = currentFrameTime;
                sendTextMessage(MESSAGE_EVENT_ADVANCE, "Your questlog has been updated.");
            }
        }
    } else {
        storageMap.erase(key);
    }
}
To the following:
C++:
void Player::addStorageValue(const uint32_t key, const int32_t value, const bool isLogin/* = false*/)
{
    if (IS_IN_KEYRANGE(key, RESERVED_RANGE)) {
        if (IS_IN_KEYRANGE(key, OUTFITS_RANGE)) {
            outfits.emplace_back(
                value >> 16,
                value & 0xFF
            );
            return;
        } else if (IS_IN_KEYRANGE(key, MOUNTS_RANGE)) {
            // do nothing
        } else {
            std::cout << "Warning: unknown reserved key: " << key << " player: " << getName() << std::endl;
            return;
        }
    }


    int32_t oldValue;
    getStorageValue(key, oldValue);

    storageMap[key] = value;
    changedStorageMap[key] = value;

    if (!isLogin) {
        auto currentFrameTime = g_dispatcher.getDispatcherCycle();
        if (lastQuestlogUpdate != currentFrameTime && g_game.quests.isQuestStorage(key, value, oldValue)) {
            lastQuestlogUpdate = currentFrameTime;
            sendTextMessage(MESSAGE_EVENT_ADVANCE, "Your questlog has been updated.");
        }
    }
}

And last of all in iologindata.cpp change:
C++:
    if (!db.executeQuery(fmt::format("DELETE FROM `player_storage` WHERE `player_id` = {:d}", player->getGUID()))) {
        return false;
    }
To:
C++:
    /*if (!db.executeQuery(fmt::format("DELETE FROM `player_storage` WHERE `player_id` = {:d}", player->getGUID()))) {
        return false;
    } //We commented this out since we are no longer using it. */

and change:
C++:
    for (const auto& it : player->storageMap) {
        if (!storageQuery.addRow(fmt::format("{:d}, {:d}, {:d}", player->getGUID(), it.first, it.second))) {
            return false;
        }
    }
  
    if (!storageQuery.execute()) {
        return false;
    }
To:
C++:
    for (const auto& it : player->changedStorageMap) {
        if (!storageQuery.addRow(fmt::format("{:d}, {:d}, {:d}", player->getGUID(), it.first, it.second))) {
            return false;
        }
    }
  
    storageQuery.addEnd(" ON DUPLICATE KEY UPDATE `value` = VALUES(value)");

    if (!storageQuery.execute()) {
        return false;
    }


Warning: One change this lazy way of doings this will cause is that no storage value will ever be removed, you can change it to -1 for "false" if you are looking to "delete" a storage value, but it won't actually delete it from the sql. Which shouldn't be a problem for 99% of all servers, since almost no one deletes a storage value.
 
Last edited:

oen432

Legendary OT User
Joined
Oct 3, 2014
Messages
1,452
Solutions
49
Reaction score
1,545
Location
Poland
GitHub
Oen44
Just save in binary instead of hundreds of rows per player and add that as a column in players table. Same thing can be done with items, depot items and so on.
 
OP
OP
Flatlander

Flatlander

Species Developer
Premium User
Joined
Feb 17, 2009
Messages
2,459
Solutions
3
Reaction score
1,342
Location
Texas
Just save in binary instead of hundreds of rows per player and add that as a column in players table. Same thing can be done with items, depot items and so on.

Like I said, this is the lazy way of doing it. Takes a few lines of code. If you want to do it some other way go for it. I'm not going to stop you.
 

skulls

Well-Known Member
Joined
Aug 28, 2010
Messages
46
Reaction score
71
Nice improvement.

I wouldn't store the -1 in the main storage map though, I'd still delete it, since it's just waste of memory, but not a big deal.

Also, if you're really concerned about having those -1 stored in the database you can easily run a "delete blablabla where value = -1".

It's also nice to highlight that this is a memory trade off (shouldn't make much difference for 2M storages, since it's only 8Mb to store 2M uint,int pairs, but for hundred millions it become to be relevant).
 
Top