OTClient Graphics Discussion (Anyone out there more knowledgeable than me?)

[Flatlander]

First, I would like to provide a video where I tested some of the FPS Dropping issues on the OTClient:

From other conversations and forum threads, it has been suggested the OTClient has the following issues:

• Large Textures: The way the OTClient draws animated items, and outfits is bad.
  
• Bad Lighting System: The way the OTClient draws lighting effects is bad.
  
• Bad Screen Drawing System: I've looked into this myself, when on the flash client, or Tibia 11 Client I notice that the client draws all of one texture at the same time.

Below under the spoilers I explain each issue in more detail to the best of my knowledge:

Large Textures:

Spoiler: Large Textures

In regards to Animated items and Outfits:
I was told the OTClient currently works like this: (I have not verified this, but it sounds reasonable due to how the frames drop when displaying animated items or outfits)

When the OTClient has to draw an animated item, currently it saves the entire animation as one giant texture, and sends this to the GPU to be drawn, then the GPU draws the one frame of that animation needed for the screen.

What this means, is if you have an item with 5 animation frames, it sends the entire texture to the GPU each frame. (item is 32x32 pixels, 5 frames means 32x32x5=5120 pixel sized texture sent to GPU each frame)

Now, most computers can handle this, but the issue is when you start having multiple creatures with the smooth walking animation. (7 frame walking animation, players are 64x64 pixels, and there are 4 directions, so the OTClient makes a huge texture of 64x64x7x4=114688 pixels)

It then sends this texture to the GPU EVERY SINGLE FRAME. this doesn't count add-ons which is a separate template.

I was told this should be changed, so instead of making one giant texture per outfit, it should make a library of textures, then send the one needed for that frame, which would reduce the pixels sent to GPU per outfit per frame from 114688 down to 4096. (28x less pixels)

I'm not sure if this is true, but from what I have looked into by reading the OTClient source code it SEEMS true.

Bad Lighting System:

Spoiler: Bad Lighting System

From another post, it was suggested by Shadowsong that the best way to do the lighting would be using Vertex Shaders. But also he stated he changed the Necronia Client to use Fragment Shaders, then also reduced the amount of lights on screen to increase FPS.

I find that reducing the size of lights is the main thing that helps reduce frame rate drops.
I above in the video I created as a test, you can see that the animated effects with lighting cause more FPS drops than the animated effects.

Bad Screen Drawing System:

Spoiler: Bad Screen Drawing System

This is all from looking at how Tibia 11 client and the flash client draws items. It seems when these newer clients pull up a texture, it will use it to draw all of that item on the screen before going to the next texture.

In the following video, you can see (when the character teleports mainly) that the drawing system works as I stated above.

Here is a screenshot from 54 seconds into the video, where it is attempting to draw an entire new screen right after teleporting:

If you have any information or ideas, regarding optimization to the graphics drawing capabilities of the OTClient, please share them here.

Also, if you think you could help with fixing any of the above issues, or know of tutorials on other websites that show how it should be done, post them here.

 

[Okke]

WTF DID I JUST SEE?

 

[whitevo]

god damit this is interesting topic, sadly I have no knowledge in such things. Still going to try hard and google something up this weekend.
I have some kind of logic how it should work, but not sure is it even possible, anyway gona educate myself with google help before I suggest something useless.

 

[Nothxbye]

How normal tibia client works if you do same tests on it? Also remember you added more tiles to game window this may causes greater fps drop.

 

[Flatlander]

How normal tibia client works if you do same tests on it? Also remember you added more tiles to game window this may causes greater fps drop.

The normal tibia client is way more optimized than the OTClient. (Some client versions are better than others)

 

[Printer]

Great, that someone is trying to fix the fps drop issue. Since i have not being using otclient, i have no information to share.

 

[Flatlander]

I am going to start trying to solve the first issue (Textures being saved as huge images)

I did notice that getTexture in thingtype.cpp might be where these huge images are created.

Spoiler: thingtype.cpp getTexture

const TexturePtr& ThingType::getTexture(int animationPhase)
{
  TexturePtr& animationPhaseTexture = m_textures[animationPhase];
  if(!animationPhaseTexture) {
  bool useCustomImage = false;
  if(animationPhase == 0 && !m_customImage.empty())
  useCustomImage = true;

  // we don't need layers in common items, they will be pre-drawn
  int textureLayers = 1;
  int numLayers = m_layers;
  if(m_category == ThingCategoryCreature && numLayers >= 2) {
  // 5 layers: outfit base, red mask, green mask, blue mask, yellow mask
  textureLayers = 5;
  numLayers = 5;
  }

  int indexSize = textureLayers * m_numPatternX * m_numPatternY * m_numPatternZ;
  Size textureSize = getBestTextureDimension(m_size.width(), m_size.height(), indexSize);
  ImagePtr fullImage;

  if(useCustomImage)
  fullImage = Image::load(m_customImage);
  else
  fullImage = ImagePtr(new Image(textureSize * Otc::TILE_PIXELS));

  m_texturesFramesRects[animationPhase].resize(indexSize);
  m_texturesFramesOriginRects[animationPhase].resize(indexSize);
  m_texturesFramesOffsets[animationPhase].resize(indexSize);

  for(int z = 0; z < m_numPatternZ; ++z) {
  for(int y = 0; y < m_numPatternY; ++y) {
  for(int x = 0; x < m_numPatternX; ++x) {
  for(int l = 0; l < numLayers; ++l) {
  bool spriteMask = (m_category == ThingCategoryCreature && l > 0);
  int frameIndex = getTextureIndex(l % textureLayers, x, y, z);
  Point framePos = Point(frameIndex % (textureSize.width() / m_size.width()) * m_size.width(),
  frameIndex / (textureSize.width() / m_size.width()) * m_size.height()) * Otc::TILE_PIXELS;

  if(!useCustomImage) {
  for(int h = 0; h < m_size.height(); ++h) {
  for(int w = 0; w < m_size.width(); ++w) {
  uint spriteIndex = getSpriteIndex(w, h, spriteMask ? 1 : l, x, y, z, animationPhase);
  ImagePtr spriteImage = g_sprites.getSpriteImage(m_spritesIndex[spriteIndex]);
  if(spriteImage) {
  if(spriteMask) {
  static Color maskColors[] = { Color::red, Color::green, Color::blue, Color::yellow };
  spriteImage->overwriteMask(maskColors[l - 1]);
  }
  Point spritePos = Point(m_size.width()  - w - 1,
  m_size.height() - h - 1) * Otc::TILE_PIXELS;

  fullImage->blit(framePos + spritePos, spriteImage);
  }
  }
  }
  }

  Rect drawRect(framePos + Point(m_size.width(), m_size.height()) * Otc::TILE_PIXELS - Point(1,1), framePos);
  for(int x = framePos.x; x < framePos.x + m_size.width() * Otc::TILE_PIXELS; ++x) {
  for(int y = framePos.y; y < framePos.y + m_size.height() * Otc::TILE_PIXELS; ++y) {
  uint8 *p = fullImage->getPixel(x,y);
  if(p[3] != 0x00) {
  drawRect.setTop  (std::min<int>(y, (int)drawRect.top()));
  drawRect.setLeft  (std::min<int>(x, (int)drawRect.left()));
  drawRect.setBottom(std::max<int>(y, (int)drawRect.bottom()));
  drawRect.setRight (std::max<int>(x, (int)drawRect.right()));
  }
  }
  }

  m_texturesFramesRects[animationPhase][frameIndex] = drawRect;
  m_texturesFramesOriginRects[animationPhase][frameIndex] = Rect(framePos, Size(m_size.width(), m_size.height()) * Otc::TILE_PIXELS);
  m_texturesFramesOffsets[animationPhase][frameIndex] = drawRect.topLeft() - framePos;
  }
  }
  }
  }
  animationPhaseTexture = TexturePtr(new Texture(fullImage, true));
  animationPhaseTexture->setSmooth(true);
  }
  return animationPhaseTexture;
}

it Defines

int indexSize = textureLayers * m_numPatternX * m_numPatternY * m_numPatternZ;

then

Size textureSize = getBestTextureDimension(m_size.width(), m_size.height(), indexSize);

Seems it is going to create a giant picture called "fullImage" and place all the sprites into one image here.

fullImage = ImagePtr(new Image(textureSize * Otc::TILE_PIXELS));

Then it does a huge repeating script grabbing sprite images and placing them into the fullImage. (I think)

Can anyone who is knowledgeable on this confirm that a giant texture is being created here? (I think it is, but i'm not 100% sure)
And I guess if the answer is yes, we would need to re-script this to only give the small image that actually needs to be drawn.

 

[Flatlander]

Sorry for the double-post, I had originally wrote a 10,000 + Character post that required 2 replies.

Any help tackling the issue with Giant Textures Causing FPS Drops would be greatly appreciated.

 

[Yamaken]

Sorry for the double-post, I had originally wrote a 10,000 + Character post that required 2 replies.

Any help tackling the issue with Giant Textures Causing FPS Drops would be greatly appreciated.

I think its not even a issue, it was a optimization. It does load that frame texture one time to the GPU and then it uses a ID to render it. Its a optimization, the texture is already on the gpu the only thing the gpu do need to is to take the ID(texture ref) and render. Its not supposed to upload the giant texture each time it does render a object.

But i have saw something that does not make sense. The new created TexturePtr from ThingType::getTexture is not stored anywhere nor it shared between the same items, outfits, effects. So we need a texture manager to cache the textures? Or at least cache the ImagePtr with the frame? Another strange thing is, the last time i have use OTClient it was taking a lot of memory, like it was caching textures/frames but everything i have saw today contradicts this lol.

Another thing, std::vector<TexturePtr> m_textures in ThingType class is not being used for nothing. It was supposed to be the cache of the textures?

 

[Flatlander]

I think its not even a issue, it was a optimization. It does load that frame texture one time to the GPU and then it uses a ID to render it. Its a optimization, the texture is already on the gpu the only thing the gpu do need to is to take the ID(texture ref) and render. Its not supposed to upload the giant texture each time it does render a object.

But i have saw something that does not make sense. The new created TexturePtr from ThingType::getTexture is not stored anywhere nor it shared between the same items, outfits, effects. So we need a texture manager to cache the textures? Or at least cache the ImagePtr with the frame? Another strange thing is, the last time i have use OTClient it was taking a lot of memory, like it was caching textures/frames but everything i have saw today contradicts this lol.

Another thing, std::vector<TexturePtr> m_textures in ThingType class is not being used for nothing. It was supposed to be the cache of the textures?

Here is my current issue with the client.

1. If I fill the screen with 50 32x32 pixel items, I have a small amount of FPS Drop.
   
2. If I fill the screen with 50 32x32 pixel animated items, I have much more FPS Drop.
3. If I fill the screen with 50 creatures, my entire screen is nearly frozen and I can barely move.

So, a 50 32x32 pixel items like a bow. Can cover every tile of your screen, and you experience a slight FPS drop.
Then you delete those 50 bows, and fill your screen with fire swords. Your FPS drops much more.
Then you delete those 50 fire swords, and fill your screen with 50 bats. Your FPS drops a HUGE amount more.

Now, saving the texture in a cache, so it can be used next time the same item appears elsewhere, would increase the performance when covering your screen with fire swords, or bats.
But if you filled your screen with 50 DIFFERENT animated items. or 50 DIFFERENT creatures, would we have the same issue we are currently having?

I 100% agree that we SHOULD cache the texture, but I do not know if this is the root problem.
BTW, I may just be being ignorant and misunderstood you, so if I did, please correct me. I am trying to learn.

 

[Yamaken]

@Flatlander can you join the otland discord? https://discord.gg/014c72lqphZCNHVuJ

Another thing, can you test a really small optimization?

here after this line: https://github.com/edubart/otclient/blob/master/src/client/thingtype.cpp#L492

add this:

m_textures[animationPhase] = animationPhaseTexture;

Now on TexturePtr& ThingType::getTexture function it will get the already loaded texture instead of loading it from the spr, creating a big image and loading it into the gpu.

@edit

Creature::internalDrawOutfit is not well optimized in my opinion. It should create a complete texture and then reuse it with the outfit colors etc, if the creature got a outfit change then rebuild the texture.

 

[strutZ]

Would love to be using OtClient.. this is the only reason i'm still with tibia client..

 

[Flatlander]

@Flatlander can you join the otland discord? https://discord.gg/014c72lqphZCNHVuJ

Another thing, can you test a really small optimization?

here after this line: https://github.com/edubart/otclient/blob/master/src/client/thingtype.cpp#L492

add this:

m_textures[animationPhase] = animationPhaseTexture;

Now on TexturePtr& ThingType::getTexture function it will get the already loaded texture instead of loading it from the spr, creating a big image and loading it into the gpu.

@edit

Creature::internalDrawOutfit is not well optimized in my opinion. It should create a complete texture and then reuse it with the outfit colors etc, if the creature got a outfit change then rebuild the texture.

I have ordered a new hard drive for my PC that I use to compile. I'll have it by this weekend.

I will compile the current OTClient, and then an OTClient with your changes. Then design a server that does FPS Testing when you log in (automatic without the need of me moving)
Then I will make a video showing my results for you .

I should receive my new hard drive on Friday (tomorrow)

Also, I joined the OTLand Discord

 

[Marcelo Druida]

Another thing, can you test a really small optimization?

I had a fps DEcrease.
ofc it was not a well elaborated test but with no edits my fps was never never bellow 40, while it constantly goes to 20 with this edit.

 

[whitevo]

O boy trying to find something what I could wrap my ahead around was actually quite difficult.

In the end I rly had to go back to the very basics to even understand in fundamental level what is rendering xD
Best read I found was this on page 380: Rasterization

I started looking some unity related videos what did kinda look similar and seeing the process from nothing to product gave some ideas what could be wrong with OTC.
Once again I still know nothing about c++ and don't even understand how in what order the current otc renders objects.

My first question would be. If you use big sprites like "demon". Does it render the unvisible objects/animations what are behind its body?

About light effects I have an idea, not sure is it even possible to add to OTC.

Spoiler

The light is rendered on top of all the objects as mask. when everything else is done.
It will loop trough the full client screen+3 tiles.
it will access the tile and look for flag: emitLight.
Tiles will have a new flag: emitLigh(section[color[INT, INT, INT], strength INT], section[color[INT, INT, INT], strength INT], section[color[INT, INT, INT], Str INT], section[color[INT, INT, INT], Str INT )
- color is the typical red, green blue values
- strength is the mask value 100% would mean it will literally be 1 color (that would give us the power to actually draw texts or do lighting effects without using sprites or some external module)
- section is tile divided into 4. first value in array means top-left, 2nd is top-right, 3rd is left-down.
If not all sections used the the opaque value will fade to 0% from the outer corner to middle of tile. (if strength was not 100%)
If all sections used and the strength is quite low, the fade to 0% will start from middle to tile corners.

Now you might be wondering, how are the nearby tiles affected by light? They aren't no overlapping, no need to scale or extend the light source.
What happens if 2 different color lights are placed on same tile same section? We will use self made algorithm to blend the color and change the strength accordingly, the bigger is str the harder to change the final color itself.

Goal is to have only 1 light flag on single tile and no overlapping. This way we can render it all in 1 go.
(currently the problems seems to be it renders as many times over as many light sources are nearby, doesn't even have to be on the same tile)

Next we need to worry how to make flag on the area.
1 way is to add the emitLight parameters on the item object and when its created to the game it will automatically merge with the light tile has.
Obviously we want it to make it easy for ourselves so we make Lua table where key is itemID and value is emitLight parameters. (lets not move these flags to .dat .cpp .xml, etc)

Depending on object light strength, it will load up nearby tiles and based on premade strength formulas it will change nearby tile emitLight values too.
When object is moved, it will unmerge with current and nearby tiles add the light to targettile and its nearby tiles. if object is equipped target tile is the parent. (yep we could have rainbow lightning ppl walking around)

So yeah there will be some complicated formulas to make, but at least these parameters are changed on event not generated every millisecond.

Another problem would be animation effects. Would need to rethink do we want to use light on these. Because that would require lot of merge and unmerge light process when certain animation cycles only emit light. How do even interact with tile on the animation cycle would be quite demanding already.

With flying effects, we can simply load up the tiles effect what it will fly trough and with addEvent we merge and unmerge the light on tile.

 

[Flatlander]

O boy trying to find something what I could wrap my ahead around was actually quite difficult.

In the end I rly had to go back to the very basics to even understand in fundamental level what is rendering xD
Best read I found was this on page 380: Rasterization

I started looking some unity related videos what did kinda look similar and seeing the process from nothing to product gave some ideas what could be wrong with OTC.
Once again I still know nothing about c++ and don't even understand how in what order the current otc renders objects.

My first question would be. If you use big sprites like "demon". Does it render the unvisible objects/animations what are behind its body?

About light effects I have an idea, not sure is it even possible to add to OTC.

Spoiler

The light is rendered on top of all the objects as mask. when everything else is done.
It will loop trough the full client screen+3 tiles.
it will access the tile and look for flag: emitLight.
Tiles will have a new flag: emitLigh(section[color[INT, INT, INT], strength INT], section[color[INT, INT, INT], strength INT], section[color[INT, INT, INT], Str INT], section[color[INT, INT, INT], Str INT )
- color is the typical red, green blue values
- strength is the mask value 100% would mean it will literally be 1 color (that would give us the power to actually draw texts or do lighting effects without using sprites or some external module)
- section is tile divided into 4. first value in array means top-left, 2nd is top-right, 3rd is left-down.
If not all sections used the the opaque value will fade to 0% from the outer corner to middle of tile. (if strength was not 100%)
If all sections used and the strength is quite low, the fade to 0% will start from middle to tile corners.

Now you might be wondering, how are the nearby tiles affected by light? They aren't no overlapping, no need to scale or extend the light source.
What happens if 2 different color lights are placed on same tile same section? We will use self made algorithm to blend the color and change the strength accordingly, the bigger is str the harder to change the final color itself.

Goal is to have only 1 light flag on single tile and no overlapping. This way we can render it all in 1 go.
(currently the problems seems to be it renders as many times over as many light sources are nearby, doesn't even have to be on the same tile)

Next we need to worry how to make flag on the area.
1 way is to add the emitLight parameters on the item object and when its created to the game it will automatically merge with the light tile has.
Obviously we want it to make it easy for ourselves so we make Lua table where key is itemID and value is emitLight parameters. (lets not move these flags to .dat .cpp .xml, etc)

Depending on object light strength, it will load up nearby tiles and based on premade strength formulas it will change nearby tile emitLight values too.
When object is moved, it will unmerge with current and nearby tiles add the light to targettile and its nearby tiles. if object is equipped target tile is the parent. (yep we could have rainbow lightning ppl walking around)

So yeah there will be some complicated formulas to make, but at least these parameters are changed on event not generated every millisecond.

Another problem would be animation effects. Would need to rethink do we want to use light on these. Because that would require lot of merge and unmerge light process when certain animation cycles only emit light. How do even interact with tile on the animation cycle would be quite demanding already.

With flying effects, we can simply load up the tiles effect what it will fly trough and with addEvent we merge and unmerge the light on tile.

Per your first question, it does draw things behind characters, such as demons, but if a Tile on a floor above is blocking the tile below, it will not draw this tile. (so for example, it won't draw the floor underground below you, even though TFS may be sending the information for this floor)

As for the lighting system:
The issue is, Tibia has all these lighting effects and they work without terrible drops in FPS.

We could of course, make a tile-by-tile system, where each tile can have a maximum of 1 light source. But even when walking near Lava you get FPS drops on low-end systems.
Which means, because lava has a large red glow, and each tile emits the same large red glow, we have reduced FPS.

Currently, when you first start up the OTClient, it creates a light bubble. (A circle that slowly fades out the farther away from the center it is).
Then, each time a light is drawn, it will re-size this circle and draw it.

I wonder if we just created 10 circles (one for each light intensity size) so we did not have to re-size, would it help in reducing FPS Drops?
Also, saving a created light in cache (such as the red intensity 8 circle for lava light) so that it can be re-drawn for each tile without having to re-do any calculations would probably help.

 

[whitevo]

Per your first question, it does draw things behind characters, such as demons,

This might be first problem then.
I wonder is drawing the objects slow process or creating the object structure for videocard to draw from.

In what order the objects are drawn? (tile > items in stack order > creatures > effects?)
Can you access the pixel where a color was added?
Can you access single pixel from sprite?

If we could somehow change an order and quick check if there is point to even create/send a texture packet. Maybe it would increase the performance.

 

[margoh]

I don't know if it will be any help, but Tibia pic has file for generating lightning effect.

 

[Flatlander]

This might be first problem then.
I wonder is drawing the objects slow process or creating the object structure for videocard to draw from.

In what order the objects are drawn? (tile > items in stack order > creatures > effects?)
Can you access the pixel where a color was added?
Can you access single pixel from sprite?

If we could somehow change an order and quick check if there is point to even create/send a texture packet. Maybe it would increase the performance.

Well, you aren't wrong.
If we reduce the amount of times the OTClient uses the draw() function to draw a sprite, it would increase FPS (naturally).
The issue is, would the check, take more time than just drawing the item.

Currently it goes row by row drawing each tile, starting at the ground tiles, then it goes through all the items, then the creatures, then animations, then lighting, then default chat text.

 

[whitevo]

The issue is, would the check, take more time than just drawing the item.

Yep that's what I'm wondering too, but I guess that we can only see when you try it?

Currently it goes row by row drawing each tile, starting at the ground tiles, then it goes through all the items, then the creatures, then animations, then lighting, then default chat text.

Can you reverse the order?(but leave lighting and text to work as they work right now)
And row by row should start from last tile and work backwards (because of 64x64 sprites can cover tiles what are positions [-1,-1])

What about these questions?

Can you access the pixel where a color was added?
Can you access single pixel from sprite?

If we can check at the pixel level we would create perfect results.

Another idea is straight out ignore objects what are behind bigger objects, but problem is that lot of mappers use combination of different items to create the scenario.

1 more idea what you could already produce would be to:
Ignore an object what has same type or id (in case of monster we use type, we only take the top creature of same type)(walking from tile to tile from same position by same monster type might start looking glitchy though)
in case of items we take the top item of same id. (although we need to consider stack amount on some items too then, some sprite change on different counts..)
in case of effecst we use a limit of enums on same position, lets say 10.
Reason you had huge fps drop was you created like hundreds of animations. and each position had like ~50 effects moving on it.