RetroPlayer Shader Support - GSoC 2017

Video Demonstration

Before we start, here’s a short preview of the current state, showing a few presets. The thumbnail screen is a work in progress.

Proposal

My initial proposal was about implementing shader support in Kodi’s RetroPlayer as well as a variety of default shaders to go with it.

However, quite early on, it was suggested to me that I try something more ambitious and implement the same spec that RetroArch shaders use. This spec essentially introduces shader “preset” files: configuration files which allow for multi-pass shader configurations to achieve advanced filtering, without manually copying and pasting shader code all over the place. They have other features too, but this is their core functionality and reason of existence.

The biggest advantage of implementing this is be the vast library of pre-existing shaders written in this format, both for OpenGL (ES), and Direct3D.

Here’s a tiny selection of examples from the RetroArch page linked above:

Thus, I changed my proposal accordingly and begun research.

Challenges

• To give a bit of background, it is important to mention that - initially - RetroPlayer didn’t have its own renderer. All of its rendering was entirely reliant on VideoPlayer (the backend and renderer used when playing any kind of video). VideoPlayer’s goals are very different than RetroPlayer’s and using it has always been a temporary solution. Thus, it was clear since the beginning that I would need to get around this problem one way or the other.

• Another big challenge was discovering that the Cg shaders available turned out to not be directly compatible with HLSL. This required some porting work and infrastructure to allow for both Cg and HLSL/FX11 to work at the same time (using macros). Me and hunterk (from libretro) worked on that.

• Figuring out how to render scaling modes with scaling multipliers for multi-pass presets was tough, I got “stuck” on that for about 3 days.

Work

Renderer

On the first month, I spend my little time between studying for finals in getting acquainted with the code base. It was apparent that I’d need to get around the renderer issue, so I started working on a new renderer.

It worked, but we soon realized that maintaining all this copied code would be a nightmare, so this never got merged.

This was just the first step though. It was suggested to me because it was the easiest way to attempt something like that without being familiar with the code. Besides, it made me familiar with key pieces of the code base I’d need later on.

Fast forward a couple of months and RetroPlayer has its own renderer! I started it, but Garrett/garbear (my mentor) did most of the work for it.

Closed Pull Request

Binary Add-on

Throughout the past 2 months I’ve been working on binary add-on used to parse preset files.

Kodi interacts with the add-on through a C API, passes it through a handful of more layers of abstraction and the result is everything that’s needed to load shader presets. The preset configuration and options, shader source and LUTs/texture paths.

There’s also an .xml file that specifies a few presets along with user-friendly names/categories and short descriptions.

This is for the more basic usage, ideally we’d allow more advanced users to use any preset they want.

Add-on repo

Libretro Shaders Repo

As mentioned above, it was discovered that the Cg shaders available turned out to not be directly compatible with HLSL, so work went towards porting them. After discussions with the libretro team they agreed to help port them to HLSL/FX, as it would benefit both projects.

Regarding this, I’ve made these 14 commits (so far) to the libretro Cg/HLSL shaders repository.

Common-shaders repo commits

Spec Implementation

For something that was the “essence” of my proposal, I started this pretty late, but I’m pleased with the progress in such short time. The majority of the spec, as well as a bunch of non-documented and non-standardized features that a lot of shader presets use are implemented.

The downside is that I’ve only been working on the Direct3D 11 renderer (Windows only). The past few days, however, I’ve been abstracting away the backend-specific stuff to allow for the other renderers like OpenGL (ES). Things are looking neat on that end, everything is behind interfaces now and rendering back-ends for other platforms shouldn’t be too hard to implement.

Work-In-Progress PR

Future Work

Short-term, what’s left to do now is finish up thumbnail rendering. We can theoretically already do real-time thumbnail rendering of multiple presets on the screen, but the solution needs to scale to underpowered embedded platforms so we’ve been discussing ways to go about it.

Other than that, OpenGL and OpenGL ES backends should be implemented. I have the feeling that things will be more straightforward than Direct3D, but I don’t want to speak too soon, we’ll see.

I’ll keep contributing to this after GSoC, at least until it gets merged to master.

Acknowledgements

• I’d like to thank my mentor, Garrett Brown (“garbear”) for helping me throughout the project, either by answering my dumb questions or providing patches to my code. (also, thanks for RetroPlayer!)

• I’d like to thank the libretro/RetroArch team, especially hunterk from #libretro on Freenode for working with me on porting shaders to HLSL. Shoutout to his blog, lots of shader goodness in there.

• I’d like to thank FernetMenta, achimturan, afedchin, rechi, Nathan Betzen and the rest of the team for answering questions.

Closing thoughts

I’ve contributed to open-source software many times before, but this experience was really something else! I learned a ton in just 3 months.

My first GSoC was a wild ride - it’s safe to say that I’ll do it again, if possible, next year!