[DISCUSSION] 128x128 pixels Tibia sprites with AI (neural network)

Gesior.pl

Mega Noob&LOL 2012
Senator
Premium User
Joined
Sep 18, 2007
Messages
2,304
Solutions
38
Reaction score
1,685
Location
Poland
GitHub
gesior
There was some thread about 128x128px sprites in OTC and also some about neural network that increase images resolution 'like human' (draw like human), but I could not find it, so I created new.
------------------------------------
I decided to test how fast it is and how does it look like.

1. Can we render resized image with neural network in real time in OTC?
No.

Neural network is too slow to generate image in real time. We can't add it to OTC as part of 'rendering'.
It takes around 1 second to increase quality of 1 HD image on modern PC.

2. Can we resize items in Tibia.spr and render high resolution sprites in OTC? Yes.

2.0 What I've done.

Spent around 2 hours on neural network (NN) installation - GitHub - nagadomi/waifu2x: Image Super-Resolution for Anime-Style Art and all libraries it requires.
"waifu2x" is neural network that resize images to 2 times bigger images. I learnt how to do this by 'watching' resized anime (japan comics) images.

I resized all Tibia 8.60 sprites (32x32) to 64x64.
Then I took resized 64x64 images 'as base images' and resized it to 128x128.

2.1 How long it took?
I ran it on [email protected] with GF 1050 using CUDA (graphic card).

Resize of 11000 images from 32x32 to 64x64 took around 4 minutes. Then it took around 3 minutes to run PHP script that clean images from 'pink' background.
Resize of 11000 images from 64x64 to 128x128 took around 9 minutes. Then it took around 3 minutes to run PHP script that clean images from 'pink' background.

2.2 RESULTS
I made website where you can compare by yourself 'how it looks like' (11000 images):
EDIT:
I added images generated by NN learned on Tibia images and fixed 'Anime' version (removed 'pink transparent color').
http://vpn.skalski.pro/items_nn_anime.htm
1) NN learned using Anime images data vs simple resize
2) NN learned using Tibia images data vs simple resize
3) NN learned using Anime vs using Tibia images data

I think that 'Anime' images look better. On some images made by NN learned by Tibia images are some artifacts.
You can compare it by yourself on page '3' with Tibia vs. Anime images.


2.3 TODO / plans
- Write/find some filter that will reduce black border width and fix all items borders which are not smooth
- Rewrite some parts of OTC to make it:
- - - load sprites with configurable size (32, 64, 128, 256)
- - - load sprites from .png files (not .spr file), it will be slow, but good for development stage

What do you think about it?
Does these images look good?

Is it time for 4K Tibia?
Worth to continue work on it? (Anyone interested in OTC programming with me?)
 
Last edited:

Nigtwisj

Some person
Joined
Oct 26, 2014
Messages
124
Solutions
1
Reaction score
72
Yeah link is dead for me aswell, would like to have a look at it for sure.
 

MagicWall

Veteran OT User
Joined
Oct 12, 2008
Messages
111
Solutions
5
Reaction score
291

conde2

Active Member
Joined
Jun 22, 2008
Messages
347
Reaction score
42
Really great idea, the only problem I see is that OTC needs a rework on drawing system, it is slow and should be improved. Maybe we could do both, rework on the drawing system and add support for other sprites sizes.

Also why didn't you removed the pink background before resizing ?
 

_ferrari_

Well-Known Member
Joined
Sep 17, 2011
Messages
119
Reaction score
55
This is very exciting, I'm willing to partner up. If you are interested please contact me.
 
OP
Gesior.pl

Gesior.pl

Mega Noob&LOL 2012
Senator
Premium User
Joined
Sep 18, 2007
Messages
2,304
Solutions
38
Reaction score
1,685
Location
Poland
GitHub
gesior
Really great idea, the only problem I see is that OTC needs a rework on drawing system, it is slow and should be improved. Maybe we could do both, rework on the drawing system and add support for other sprites sizes.

Also why didn't you removed the pink background before resizing ?
I had problem, because 'transparency' is not simple thing in images. In case of item images GIFs, it detected that background is black?! I switched it to pink, as this color is never used in OTS images.
I just fixed it by converting GIF to PNG before I put it into AI.


Today I will try to learn it with OTS images to make it understand 'tibia drawing style'.

EDIT:
Right now I'm generating 100.000 images 640x640 (20x20 tiles) of random items with transparent backgrounds. I will leave my PC at work for a weekend (~60 hours) to 'learn' these images. GeForce 1050 isn't fast, but should learn to resize it better then 'anime art' network model.

I'm pretty sure that other neural network - not waifu2x with unknown network structure - would work better for Tibia images. I tried to learn it with 32x32 images, but it returned error that images are too small and minimal size is 80. Then I tried with 128x128 and it also failed.
It means that network is quite big and waste most of 'power' to calculate pixels that are not in 32x32 image (network must always process same size of image, when image is bigger script split it into few images and process one after another, when image is smaller script add empty pixels and cut them off from 'output image').
 
Last edited:

samco

4x4 Developer.
Joined
Jul 3, 2007
Messages
936
Solutions
7
Reaction score
205
Location
Spain
Only looking at the current results, the main "problem" i see is the dark border. Its way too big now :)
 

Keal Atis

Intermediate OT User
Joined
Feb 1, 2017
Messages
215
Solutions
1
Reaction score
124
64x64 looks fine, but 128x128 and 256x256 need more improvment.
Also you should do something with black square borders - maybe simple filter used on scaled image.
 
OP
Gesior.pl

Gesior.pl

Mega Noob&LOL 2012
Senator
Premium User
Joined
Sep 18, 2007
Messages
2,304
Solutions
38
Reaction score
1,685
Location
Poland
GitHub
gesior
My PC is learning now tibia-style. We will see on monday, if it will be better.
I ran some item size filter after generation to reduce size of image to original image (change pixels to transparent).
I will write some filter to find items with border and reduce it.

64x64 looks fine, but 128x128 and 256x256 need more improvment.
Also you should do something with black square borders - maybe simple filter used on scaled image.
128x128 is NN image, 256x256 is 128x128 image just resized to 256x256 without NN.
 
Last edited:

Il Knight

Veteran OT User
Joined
Dec 1, 2014
Messages
628
Solutions
7
Reaction score
321
Location
Spain
My PC is learning now tibia-style. We will see on monday, if it will be better.
I ran some item size filter after generation to reduce size of image to original image (change pixels to transparent).
I will write some filter to find items with border and reduce it.
all this is nice keep it up
 

admkhbr

Member
Joined
Mar 24, 2014
Messages
31
Reaction score
17
I must admit that I'm surprised how neural networks are amazing. I always tought it was just some new bullshit, but after searching just a few (and seeing this topic), I realized how fantastic is this technology.

@TOPIC
I liked the results, they're good. Of course, there are a little problems, like the outline, but, from myself, i think it's a little charm.

Also, both ideas for OTC are excellent. I really think sprites size is a limitation of OTC. With these features, i'm pretty sure Open Tibia will be one more step closer to be one of bests engines for 2D MMORPGs, since 32px sprites' size do are a limitation. Even though the results of neural network are not perfect, they're good enough to provide a good development environment for new games and arts with bigger sprites.

TL;DR: Keep it up, it's going to be fantastic.
 
Last edited:
OP
Gesior.pl

Gesior.pl

Mega Noob&LOL 2012
Senator
Premium User
Joined
Sep 18, 2007
Messages
2,304
Solutions
38
Reaction score
1,685
Location
Poland
GitHub
gesior
After weekend of learning NN with Tibia images (GPU processed around 250.000.000 sprites to learn 'tibia style'), test returns that 16x16 to 32x32 resize gives only 2 wrong pixels per image (0.2% wrong pixel colors).
Images look quite good, but there are some artifacts on around 10% images which make it look worse then 'Anime' style. Links in first post are updated.
Results:

1) NN learned using Anime images data vs simple resize
2) NN learned using Tibia images data vs simple resize
3) NN learned using Anime vs using Tibia images data <-- compare Anime NN, Tibia NN and simple 'resize' by Tibia client

I think that 'Anime' images look better. On some images made by NN learned by Tibia images are some artifacts.
You can compare it by yourself on page '3' with Tibia vs. Anime images.


I must admit that I'm surprised how neural networks are amazing. I always tought it was just some new bullshit, but after searching just a few (and seeing this topic), I realized how fantastic is this technology.

@TOPIC
I liked the results, they're good. Of course, there are a little problems, like the outline, but, from myself, i think it's a little charm.

Also, both ideas for OTC are excellent. I really think sprites size is a limitation of OTC. With these features, i'm pretty sure Open Tibia will be one more step closer to be one of bests engines for 2D MMORPGs, since 32px sprites' size do are a limitation. Even though the results of neural network are not perfect, they're good enough to provide a good development environment for new games and arts with bigger sprites.

TL;DR: Keep it up, it's going to be fantastic.
What AI can do when you spent milions $ on it?
Get 2 images and draw first in 'style' of second:
Deep Dream Generator


EDIT:
Added 256x256 px renders from neural network with Tibia and Anime data.
It looks like 'Tibia NN' images are more sharp, but also contrast is increased and there are some 'white holes' in them.

Removing 'black border' idea:
Detect it and remove from images before 'increasing size', list items with border. Add 1-2px border to listed items after resizing.

EDIT 2:
Items generated from 32x32 to 256x256 with no border:
2412.png

2413.png

2415.png

2416.png

2417.png

2418.png

2419.png

2420.png

2421.png

2422.png

2428.png

12654.png
 
Last edited:

Znote

<?php echo $title; ?>
Staff member
Global Moderator
Premium User
Joined
Feb 14, 2008
Messages
6,871
Solutions
243
Reaction score
1,716
Location
Norway
GitHub
Znote
This looks awesome!
Never though NN techniques could be applied to low res image upscaling, the anime image taugh versions looks surprisingly well. :eek: I wonder why that is. Because anime photos tend to stick to fewer color shades and smoother lines, making the output overall more consistent and smooth?

Edit: Realized this has Nvidia cuda dependency. My red team monster ain't gonna work well.
 
Last edited:

4drik

Advanced OT User
Joined
Jun 30, 2014
Messages
236
Reaction score
178
H32LnYR.png

In Photoshop we can click ALT+CTRL+R and adjust the edges by moving with two sliders.

#edit:
@Gesior.pl
Result:
H33RO1s.png
 
Last edited:
OP
Gesior.pl

Gesior.pl

Mega Noob&LOL 2012
Senator
Premium User
Joined
Sep 18, 2007
Messages
2,304
Solutions
38
Reaction score
1,685
Location
Poland
GitHub
gesior
This looks awesome!
Never though NN techniques could be applied to low res image upscaling, the anime image taugh versions looks surprisingly well. :eek: I wonder why that is. Because anime photos tend to stick to fewer color shades and smoother lines, making the output overall more consistent and smooth?

If you want some processing power, I could help you run NN training. My desktop computer is specced with a Ryzen 1800x (8 cores 16 threads @3.9Ghz) cpu, 32GB DDR4 [email protected], with a Vega 56 gpu. Which I believe might perform very well for graphic NN operations. (Never done any of this tho so donno).
Results of 'tibia images NN' are worse then 'anime images NN'. I don't know why. I will try to contact author of this NN implementation.
I will also search for some other NN implementation that we can adjust to tibia image requirements (32x32 px input) and increase number of NN 'layers'. Current implementation process 80x80 px images and uses 1.5GB RAM + 513MB on graphic card.

This week I will try to write some border-fixing algorithm to make all items smooth, not only these with black borders. I got few simple concepts that should give nice results for most of items.

@4drik
Try it with:
http://vpn.skalski.pro/items_nb/2420.png
and post results.
 

ScorpionOT

uber n00b
Joined
Mar 28, 2015
Messages
378
Solutions
6
Reaction score
114
Gesior. I've been thinking of writing a module maker for OTC in order to implement a more efficient creation of DAT and SPR. Could this be helpful?
 
Top