Help with shader outline

[Evolunia]

Hello, I'm trying to make some shaders for otclient.
Currently I am trying to make an outline around my outfit, I've got it working nicely but some outfits are a bit buggy while walking and even some are bad stationary, but the majority seems to work flawlessy
I don't have any experience with shaders before, I played around with it now for a little bit and this is what I've made:

uniform mat4 u_Color;
varying vec2 v_TexCoord;
varying vec2 v_TexCoord2;
uniform sampler2D u_Tex0;

const float ALPHA_TOLERANCE = 0.01;

void main()
{
    vec4 baseColor = texture2D(u_Tex0, v_TexCoord);
    vec4 texcolor = texture2D(u_Tex0, v_TexCoord2);
    if(texcolor.r > 0.9) {
        baseColor *= texcolor.g > 0.9 ? u_Color[0] : u_Color[1];
    } else if(texcolor.g > 0.9) {
        baseColor *= u_Color[2];
    } else if(texcolor.b > 0.9) {
        baseColor *= u_Color[3];
    }

    vec4 pixel1 = texture2D(u_Tex0, vec2(v_TexCoord.x + 0.001, v_TexCoord.y));
    vec4 pixel2 = texture2D(u_Tex0, vec2(v_TexCoord.x - 0.001, v_TexCoord.y));
    vec4 pixel3 = texture2D(u_Tex0, vec2(v_TexCoord.x, v_TexCoord.y + 0.001));
    vec4 pixel4 = texture2D(u_Tex0, vec2(v_TexCoord.x, v_TexCoord.y - 0.001));

    bool neighbourColor = pixel4.a > ALPHA_TOLERANCE || pixel3.a > ALPHA_TOLERANCE || pixel2.a > ALPHA_TOLERANCE || pixel1.a > ALPHA_TOLERANCE;

    if (baseColor.a < ALPHA_TOLERANCE && neighbourColor) {
        baseColor.rgb = vec3(1.0, 0.0, 0.0);
        baseColor.a = 0.7;
    }
    
    gl_FragColor = baseColor;
    if(gl_FragColor.a < 0.01) discard;
}

To detect where the outline should be drawn, it checks if the current color is a transparent color and if any of the neighbouring colors is a "real" colour.
Is there some better way to achieve this?


Here's some examples on how it looks like:

As you can see the gamemaster outfit is bugged with it, and there's two red lines where there shouldn't be any. Any tips on how to fix that? I've tried a lot of different things, but I can't seem to get a nice solution for it, so now I asking here

 

[Kotetso]

What if you copied the base image, changed it for the colour of your choice, and placed it under the desired image? I've done it like that for a different project and it worked

 

[Fresh]

Framebuffer

 

[Kotetso]

@Fresh hey dude, you seem very helpful here on otland is there any chance you could help me with an issue im having regarding auras/shadders?

 

[Rexxar]

Hello, I'm trying to make some shaders for otclient.
Currently I am trying to make an outline around my outfit, I've got it working nicely but some outfits are a bit buggy while walking and even some are bad stationary, but the majority seems to work flawlessy
I don't have any experience with shaders before, I played around with it now for a little bit and this is what I've made:

uniform mat4 u_Color;
varying vec2 v_TexCoord;
varying vec2 v_TexCoord2;
uniform sampler2D u_Tex0;

const float ALPHA_TOLERANCE = 0.01;

void main()
{
    vec4 baseColor = texture2D(u_Tex0, v_TexCoord);
    vec4 texcolor = texture2D(u_Tex0, v_TexCoord2);
    if(texcolor.r > 0.9) {
        baseColor *= texcolor.g > 0.9 ? u_Color[0] : u_Color[1];
    } else if(texcolor.g > 0.9) {
        baseColor *= u_Color[2];
    } else if(texcolor.b > 0.9) {
        baseColor *= u_Color[3];
    }

    vec4 pixel1 = texture2D(u_Tex0, vec2(v_TexCoord.x + 0.001, v_TexCoord.y));
    vec4 pixel2 = texture2D(u_Tex0, vec2(v_TexCoord.x - 0.001, v_TexCoord.y));
    vec4 pixel3 = texture2D(u_Tex0, vec2(v_TexCoord.x, v_TexCoord.y + 0.001));
    vec4 pixel4 = texture2D(u_Tex0, vec2(v_TexCoord.x, v_TexCoord.y - 0.001));

    bool neighbourColor = pixel4.a > ALPHA_TOLERANCE || pixel3.a > ALPHA_TOLERANCE || pixel2.a > ALPHA_TOLERANCE || pixel1.a > ALPHA_TOLERANCE;

    if (baseColor.a < ALPHA_TOLERANCE && neighbourColor) {
        baseColor.rgb = vec3(1.0, 0.0, 0.0);
        baseColor.a = 0.7;
    }
   
    gl_FragColor = baseColor;
    if(gl_FragColor.a < 0.01) discard;
}

To detect where the outline should be drawn, it checks if the current color is a transparent color and if any of the neighbouring colors is a "real" colour.
Is there some better way to achieve this?


Here's some examples on how it looks like:

As you can see the gamemaster outfit is bugged with it, and there's two red lines where there shouldn't be any. Any tips on how to fix that? I've tried a lot of different things, but I can't seem to get a nice solution for it, so now I asking here

Try tweaking the ALPHA_TOLERANCE value up and down and see if it fixes the weird squares.

Might also be worth extracting the sprite for the gamemaster outfit and having a look at the pixels nearby where it wrongly highlights red, see what the "alpha" color actually is there so you can compensate for it.

 

[mrianura]

You have to draw whole outfit to frame buffer (which will be a bit bigger than final outfit square) and at the end draw that frame buffer with shader on the map.

 

[Evolunia]

Try tweaking the ALPHA_TOLERANCE value up and down and see if it fixes the weird squares.

Might also be worth extracting the sprite for the gamemaster outfit and having a look at the pixels nearby where it wrongly highlights red, see what the "alpha" color actually is there so you can compensate for it.

tweaking alpha_tolerance didn't help, and it's just not gamemaster, it also happens on some other outfits but just while walking. i think it's when a sprite utilizes all of it's space (?)
anyways had some time today so I tried what @Kotetso wrote and it made the code look a bit nicer. I added another red texture that's larger than the normal outfit

attribute vec2 a_TexCoord;
attribute vec2 a_Vertex;
uniform mat3 u_TransformMatrix;
uniform mat3 u_ProjectionMatrix;
uniform mat3 u_TextureMatrix;
uniform vec2 u_Offset;
uniform vec2 u_Center;
varying vec2 v_TexCoord;
varying vec2 v_TexCoord2;
varying vec2 v_TexCoord3;
void main()
{
    vec2 vertex = a_Vertex;
    vec2 mainTexture = a_TexCoord;
    vec2 outlineTexture = a_TexCoord;
    vec2 delta = vec2(sign(u_Center - (vertex)));
    vertex -= delta * vec2(1.0, 1.0);
    mainTexture -= delta * vec2(1.0, 1.0);
    gl_Position = vec4((u_ProjectionMatrix * u_TransformMatrix * vec3(vertex.xy, 1.0)).xy, 1.0, 1.0);
    v_TexCoord = (u_TextureMatrix * vec3(mainTexture, 1.0)).xy;
    v_TexCoord2 = (u_TextureMatrix * vec3(mainTexture + u_Offset, 1.0)).xy;
    v_TexCoord3 = (u_TextureMatrix * vec3(outlineTexture, 1.0)).xy;
}

uniform mat4 u_Color;
varying vec2 v_TexCoord;
varying vec2 v_TexCoord2;
varying vec2 v_TexCoord3;
uniform sampler2D u_Tex0;
const float ALPHA_TOLERANCE = 0.1;
void main()
{
    gl_FragColor = texture2D(u_Tex0, v_TexCoord);
    vec4 texcolor = texture2D(u_Tex0, v_TexCoord2);
    vec4 texcolor3 = texture2D(u_Tex0, v_TexCoord3);
    if(texcolor.r > 0.9) {
        gl_FragColor *= texcolor.g > 0.9 ? u_Color[0] : u_Color[1];
    } else if(texcolor.g > 0.9) {
        gl_FragColor *= u_Color[2];
    } else if(texcolor.b > 0.9) {
        gl_FragColor *= u_Color[3];
    }
    if (texcolor3.a > ALPHA_TOLERANCE && gl_FragColor.a < ALPHA_TOLERANCE) {
        gl_FragColor.rgb = vec3(1.0, 0.0, 0.0);
        gl_FragColor.a = 0.7;
    }
    if(gl_FragColor.a < 0.01) discard;
}

Still got the same issue as the first one though, any code examples on how to use frame buffers?

If I make the red texture a lot larger it also ends up looking like this:

will probably end up doing it like this though, adding some blur is easy now, and the few weird outfits can be fixed by adding some empty space

 

[Competitibia]

attribute vec2 a_TexCoord;
attribute vec2 a_Vertex;
uniform mat3 u_TransformMatrix;
uniform mat3 u_ProjectionMatrix;
uniform mat3 u_TextureMatrix;
uniform vec2 u_Offset;
uniform vec2 u_Center;
varying vec2 v_TexCoord;
varying vec2 v_TexCoord2;
varying vec2 v_TexCoord3;

void main()
{
    vec2 vertex = a_Vertex;
    vec2 mainTexture = a_TexCoord;
    vec2 outlineTexture = a_TexCoord + u_Offset;
    vec2 delta = vec2(sign(u_Center - (vertex)));
    vertex -= delta * vec2(1.0, 1.0);
    mainTexture -= delta * vec2(1.0, 1.0);
    gl_Position = vec4((u_ProjectionMatrix * u_TransformMatrix * vec3(vertex.xy, 1.0)).xy, 1.0, 1.0);
    v_TexCoord = (u_TextureMatrix * vec3(mainTexture, 1.0)).xy;
    v_TexCoord2 = (u_TextureMatrix * vec3(mainTexture + u_Offset, 1.0)).xy;
    v_TexCoord3 = (u_TextureMatrix * vec3(outlineTexture, 1.0)).xy;
}

let me know

uniform mat4 u_Color;
varying vec2 v_TexCoord;
varying vec2 v_TexCoord2;
varying vec2 v_TexCoord3;
uniform sampler2D u_Tex0;
const float ALPHA_TOLERANCE = 0.1;

void main()
{
    gl_FragColor = texture2D(u_Tex0, v_TexCoord);
    vec4 texcolor = texture2D(u_Tex0, v_TexCoord2);
    vec4 texcolor3 = texture2D(u_Tex0, v_TexCoord3);
    
    if(texcolor.r > 0.9) {
        gl_FragColor *= texcolor.g > 0.9 ? u_Color[0] : u_Color[1];
    } else if(texcolor.g > 0.9) {
        gl_FragColor *= u_Color[2];
    } else if(texcolor.b > 0.9) {
        gl_FragColor *= u_Color[3];
    }
    
    if (texcolor3.a > ALPHA_TOLERANCE) {
        gl_FragColor.rgb = vec3(1.0, 0.0, 0.0);
        gl_FragColor.a = 0.7;
    }
    
    if(gl_FragColor.a < 0.01) {
        discard;
    }
}

 

[felpscouto]

Hello guys! Someone can explain to me how this outline works? I want to create a similar thing in my server!