Pixel Art Shader - Feedback wanted

[Custodian]

I'm seeking feedback on some WIP sprites. I'm creating a custom pixel art shader in my 3D modeling software. I would like a general critique of the rendering quality of each sprite. Do they look convincingly like they might be hand-drawn sprites? Which of the two sets of sprites do you prefer? Which do you like the best and/or least, and why? I think the biggest giveaway that these are not hand-drawn is that the animation is clearly procedural. Any other feedback welcome. Follow the links to see the animations.

---

Which of these do you like better, and why?

https://imgur.com/2zI0QX9

https://imgur.com/N2ZCmIr

---

Which of these do you like better, and why?

https://imgur.com/DjoxG2I

https://imgur.com/NgNHxob

[Misery]

Obviously, for the first set, the top one is definitely better. It has a little bit of nice detail in it. The second one looks like it was designed for Atari. As for the second set, I think you made a mistake, 'cause they both look almost exactly the same. The only difference is the outfit, and it's barely a difference. Jus' the outfit has a few slightly different coloured pixels. But whatever or not they look "hand drawn" I have no idea. I'm not that kind of artist.

[Valken]

Top - First picture
Bottom - Last picture...

Great mix of PIXEL and details... we want both.

[Gez]

Top: while I like the look of the first picture better, for a pixel art aesthetic, maybe the lower resolution of the second picture is better.
Bottom: first picture; the second has a pixel graininess that makes it look more like a low-res digitized photo than pixel art. But I'd want to suggest giving your character a bigger head. Pixel art may need to cheat a bit with proportions, because the face is an especially important area and at this resolution it just doesn't look like a face anymore.

[Custodian]

Thanks for the feedback! This is a technical challenge. I'm attempting to devise a repeatable process to render sprites out of my animation software (Autodesk Maya) so that they will require minimal cleanup when bringing them into Doom.

I agree that a lot of the detail gets lost, especially the more crucial parts like the face. It's clear that my solution needs to retain more detail. The biggest challenge has been to get Maya to render images with a limited color palette. Converting a full color image to an indexed color image causes unwanted noise and poor color translations. I've devised a way to restrict the palette in Maya and clamp down antialiasing. I refined my process and applied it to a weapon model.

Click to see the full animation.

Every frame was rendered out of Maya. Post processing was done in Aseprite to add the muzzle flare. Those were the only frames I altered manually. I'm quite pleased with these results. Note: The palette I used is not Doom's palette. My project is using this palette.

[Caligari87]

Both examples of the character look pretty good though I prefer the ones with more details in the coloring.

If I'm being completely honest, the weapon example looks exactly like "3D model animation frames downscaled and palettized in an image editor" and not "pixel art". It's the difference between a weapon rip and redrawing the weapon.

Though that may also have something to do with the animation, I'm not sure.

I'm not saying this to be discouraging, just noting that while it's an interesting exercise in shader development, it's going to fall short of "pixel art" without some more work on the asset side.


After watching the animation some more, I think a couple big conjoined issues are the banding and the lighting. Banding is one of the biggest offenders with "pixel art" that's just crunched from a higher-res source. You need to take special care that the lighting changes are subtle, and that gradients of color are either carefully dithered, or restricted to ranges with enough color in the palette to support a smooth transition.

I think this works better on the character examples because of the limited original color palette and the flat lighting or more matte materials.

[Enjay]

I fully agree with everything that Cali said, but my biggest problem with the pistol was the position of the hand/wrist. The hand looks like it has an incredibly weak and slightly awkwardly position on the grip and the angle of the wrist looks very awkward. If a big chunky pistol was fired by someone holding it like that, they'd probably miss, might injure their wrist and would almost certainly lose their grip on the weapon. I know that this isn't to do with the pixel shader effect (like I said, Cali has that covered) but I thought it was worth mentioning because it could probably be fixed reasonably easily.

Other than that, I actually really like the look that you have gone for and the palette really works well. What you have shown so far gives me a sort of "Another World" vibe - sort of like a sequel level of graphics.

[Caligari87]

Thanks Enjay, I couldn't put a finger on what the characters reminded me of, but Another World was exactly it.

[Custodian]

This is great feedback. Thanks!

Cali hit the nail on the head. The palette I've chosen does not have a sufficient grade to capture the detail in the highlights. I think the white-to-black gradient in PLAYPAL has 32 shades of grey? Mine has just 8. That said, I'd like to avoid the 'cell shaded' look too. I also find dithering to look quite ugly unless it's done manually

For reference, here is the 3D model and disembodied arm:

---

The image below shows a few things. On the left, my model with a 'realistic' shader (It's not a PBR shader, but it does respect energy conservation so that highlights don't blow out or produce fireflies.) The next two images are simply the first, but have been translated to an indexed palette in Aseprite with the Doom and Duel palettes, respectively. The last two images on the right were rendered directly from Maya with my 'palette matching' shader... the shader that I'm trying to use to achieve a pixel-art look.

---

I'll return to this post later and see if I can extend my limited palette by adding dithering. I find dithering to be ugly, and it can get noisy on animations. But, it might be possible to achieve a nice dither in Maya. I'm going to see if I can generate something like this:

[Caligari87]

Something to consider with emulating pixel art is that true pixel art has restrictions which influence the final product. For example, since pixel art is most often drawn by hand, objects rarely rotate. If they do rotate, they almost never have things like realistic reflections, specular, angle-of-incidence lighting, etc.

I know it's tempting to try using realistic materials, but maybe try biasing things toward diffuse textures and only using limited highlights on important things like edges, not large flat planes and shallow curves. That's where the banding ends up looking worst because those are large gradients with more pixels than you have colors, whereas a small specular highlight on an edge or sharp curve will have at most the same number of pixels as your palette has colors. Does that make sense?

EDIT: The same can apply with animation. If your gun changes angle, you would expect the color on the flat side to change with that angle. See: Doom's boss cube:

If you have more frames of animation than you have color space, the color transitions between each angle are more jarring.

[Custodian]

This article in particular was really helpful. I recommend giving it a read.

@Enjay - Heard! I'm kinda using anime logic for the gun, but I agree the animation should feel more powerful. I gave it a bit more kick and tightened the character's grip.

@Cali - Fantastic recommendations! I don't think I'm going to 100% hit the hand-drawn look because the animations are always going to be a bit smoother because they're from an animation program. But, I did follow most of your suggestions.

I masked the lights per object so I could manually control the light for each surface. I made each light a child of each surface it illuminates so that the lights would move along with the objects while they animate and keep the angles the same from frame to frame. I removed all specular highlights and faked highlights by making them part of the diffuse. I also switched to a color palette with more shades per color at the cost of using fewer colors (I don't really need two different lime-greens, do I?)

I think the results look far better than my first attempt. I used a subdivision scheme with smooth tangents to ensure that even perfectly flat surfaces would have a bit of a gradient to them, and so that hard edges would catch more light. Lastly, I rendered a world-space normal map and then applied it to the camera normal of the mesh to help the illumination remain consistent while the mesh moves. The only thing I was not able to achieve was a dither effect.

Let me know what you like or dislike about the new animation below!



There is a bit of noise in the rails of the gun in the holster/draw animations. I really pumped up the samples for that geometry to try to eliminate that noise, but wasn't able to fully remove it. That's definitely something I'll need to touch up in an image editor.

My end goal is to make the sprite-to-doom pipeline a little more accessible. In the past I've spent hundreds of hours on sprite sheets drawing and tweaking images pixel-by-pixel. Thanks again for all the awesome feedback!

[Caligari87]

Definitely looks much better! Even the banded areas on the cylinder feel more intentional now, rather than being artifacts of downsampling like they were before. I don't have any further recommendations at this point but keep up the good work!

[Enjay]

Indeed, it looks much better to me too.

I do still feel that there is something awkward about the angle of the wrist though. It's not the anime style, it's just an awkward angle. I've exaggerated it in these pics (and they were awkward to take as well - holding both the gun and the camera and trying to get enough distance to get a decent pic), but the setup you have looks more like the first (very unnatural) pic to me rather than the better angled second one.

[Custodian]

Thanks Enjay! I've downloaded that image and will be using it for reference when I pose my hand and gun models. I'll post when I update.

I'll update this thread periodically with progress on some of my sprite work. If I ever get the time, I'll see if I can write up a tutorial on how I set up my 3D software to render in this style. I'm not sure where I can host a tutorial where it won't get lost to time.



As I post more progress, I definitely welcome feedback. Hopefully the community can get some use from these sprites too.

[Gez]

I'm not sure where I can host a tutorial where it won't get lost to time.

A wiki. Either the ZDoom wiki or the more general Doom Wiki (at DoomWiki.org). Both have a tutorial section where it could fit.