Preface

I’ve been contributing a lot to an emulation project that was open-sourced about a year ago by its founding developer, PetMac. As the title suggests, it aims to emulate the Playstation Vita.

When it was initially released, it booted a bunch of homebrew titles and a few months later - after me and a few other developers have been working on it - it has progressed a great deal.

As is usual with such emulation projects (see Dolphin’s, RPCS3’s, Citra’s, etc), I really wanted to have regular Progress Reports. So, I have taken on the task of writing these reports, every so often.

As the previous posts, it got some decent traction from /r/emulation and similar forums, which is great as it helps spread the word and potentially helps our Patreon, which would allow some of us to devote more time on it.

Anyway, that’s it for the preface - I’m posting the whole thing below or alternatively you can read it on our project’s blog here: https://vita3k.org/2018/09/05/Summer-2018-Progress-Report.html

Introduction

Welcome to this summer’s progress report! It’s been a while.

Progress was slow early in the summer as most devs didn’t have much time, so we decided to only do Patreon announcements and not full blog posts.

Development has picked up in late July and August though and there has been progress, with more exciting thing to come in the future. There’s too much to summarize here, so let’s get right into it!

Major Changes

First steps to SPIR-V based Shader Recompiler (#297, #300 & others) by VelocityRa

A bit about modern GPU emulation

Modern console GPUs run chip-specific machine code that is generated by compiling higher-level shader languages, when game developers deploy their games. The original high-level code is lost and what we’re left with is a blob of 1’s and 0’s that encode shader programs using proprietary encoding schemes and instruction set architectures (ISAs), specifically tailored to the specific piece of hardware they’re intended to run on (for PCs and even some consoles, the process is a little different, but this is discussion is about the Vita, so let’s not digress).

Our job as emulator developers, is to get these shader programs that contain information about how graphics data is processed and run it on any GPU that a modern PC might have.

For this to be possible, we first need to understand the ISA and the encoding of the proprietary shader language of the emulated system. When this is done, we can go ahead and create a translator for it, that targets modern shader languages. Then, we can let graphics drivers do their job of compiling our translated code to the format of whatever host GPUs the emulator is running on.

To bring things to where our work is concerned

When starting the project, we knew nothing about the ISA and the encoding of the Vita’s shader programs. Some emulators are lucky enough to come across public official documentation that helps understand the chip (AMD for example, is particularly good in releasing all kinds of information about its GPUs), but unfortunately this wasn’t the case here. The Vita’s GPU is a SGX543MP4+ made by PowerVR, a GPU designer that makes GPUs for mobile devices. They’ve gotten better at releasing docs, but for this particular (older) GPU there is no public documentation that would be helpful to us.

So the only other option is for us is to make that documentation ourselves. Understand the part of the GPU we need by reverse engineering it outselves. Developer motoharu has took on this tedious task and is bringing us closer and closer to understanding the Vita’s shaders. His work is published here.

For Vita3K, we have decided to recompile (transform) Vita shaders to a standardized intermediate shader language that is close to low-level assembly, called SPIR-V. After this, we use already existing tools to convert this language to GLSL, a higher-level language that graphics drivers understand (newer hardware understands SPIR-V too, but we wish to be as compatible as possible with older hardware).

Most modern emulators forgo this intermediate approach, in favor of directly transpiling to GLSL or similar high-level shader languages themselves, but we thought it was in our best interest to do it this way for various reasons we won’t delve into here.

This Pull Request implements very simple, preliminary/stub shader generation, using the GXP -> SPIR-V -> GLSL method described. So, while not directly affecting games yet, this work is essential for displaying anything in Vita3K, without taking ‘shortcuts’ by hardcoding shaders as we’ve been doing.

Following PRs by VelocityRa improve on this work and lots more work is expected to follow as we understand more about the Vita’s shaders and its graphics API and make use of this knowledge in the form of a recompiler.

Fixez (#309) by VelocityRa

Many graphics (libGXM emulation, shader generation, renderer) improvements, bringing us closer to shader recompilation.

That PR also included a fix for #302, which took VelocityRa, frangarcj and PetMac many hours of debugging/reversing to figure out, since we discovered there was an important part of the Vita graphics pipeline that wasn’t being handled correctly.

Thankfully, reversing libGXM (Vita’s graphics library) beared fruit and we were able to implement a working solution that is closer to how the hardware does the particular task of reserving memory for uniform buffers (part of memory used for sending data to the GPU).

Several fixes (#312) by frangarcj

Another wonderfully named PR that implements and stubs a bunch of functions used by some commercial games. One of those games isPersona 4 Golden, which this PR brought quite close to the intro video playing.

Note: There has been some new progress on this, wait to see more on the next Progress Report, or join our Discord server to be kept in the loop (almost all dev discussion happens there)!

Stub commonly used sceNgs functions (#292) by VelocityRa

SceNgs is a Vita module responsible for audio generation. Many commercial games crashed early on its functions.

This PR stubs a few of them resulting in many commercial titles (like Disgaea and Dangaronpa games, Steins;Gate, etc) progressing further and/or booting.

Notable Improvements

Implement sceIoGetstatByFd (#305) by bentokun

Implements an I/O related feature/function that fetches information about Vita files and folders, resulting in numerous improvements in homebrew (and most likely commercial) games.

Game Select: Redesign & sorting functionality (#294) by VelocityRa

Complete re-design of our game select screen to be grid-based and the addition of sorting by title ID and game title (the latter one being used by default).

Screenshot

Shader gen, kernel, IO and renderer improvements (#307) by VelocityRa

Another set of various changes, mostly graphics related, with some I/O fixes and stubs (incomplete implementations) of 10 kernel functions.

This PR makes the console/TTY output of several commercial games (P4G included) visible, providing very useful debugging information to Vita3K developers.

Config System & New Command-line Arguments (#310 & #313) by VelocityRa

This work sets up a proper config system for Vita3K and adds the option to run installed games directly from the command line without any extra steps, speeding up testing.

Another noteworthy option added is the ability to change the log level to only show log messages of specific importance, preventing log spam if desired.

Independent rendering support (#284 & #293) by VelocityRa & PetMac

We used to rely on imgui (our GUI library) to render everything. The first PR includes OpenGL screen texture rendering support so that we can do that ourselves and without displaying any GUI and the second one disables imgui rendering entirely, by using the same method to draw the game in GUI and no-GUI mode.

Cleaner/less code, easier graphics debugging, less dependance on our GUI library.

Use the “G” key to toggle GUI rendering.

Renderer refactoring (#287 & #285) by PetMac

Refactors GXM emulation and renderer components to be separate in the code base, resulting in much cleaner and more maintainable code while also improving emulation accuracy because of improved graphics state management.

Mostly module fixes (#290) by VelocityRa

Various crash fixes to the kernel and the AppUtil module, as well as improvements to our loader, which facilitates the loading and execution of Vita executables (like OS modules, or games themselves).

Progress Screenshots

We don’t have the testing manpower to indicate which change-set improved each title, so here’s a collective set of progress screenshots/videos over the summer.

Commercial Games

Progress from this month allowed these games to progress and thanks to hand-written shaders we can actually see what’s going on.

Downwell

Downwell now goes in-game! Too slow to be playable, but other than that it now plays accurately and without freezes/crashes.

Videos

Siralim

This was featured in our latest report, but a user recorded a video and as you can see, it’s almost playable.

Video

Reality Fighters

This game progressed, but crashes at the loading screen.

Screenshot

Uncanny Valley

Alone With You

The game now advances further, it shows a cut-scene and dialogue before freezing.

Screenshots

There’s many other improvements in commercial games, but we can’t show off much other than logs, since no-one has written shaders for them.

Homebrew

Here’s a few other screeshots showing homebrews that either boot or are playable for the first time after this month’s work.

Another Metroid 2 Remake (AM2R)

This well-known homebrew remake of ‘Metroid II: Return of Samus’ now goes in-game!

Screenshot

DstroyVita

This ‘Dstroy’ port now boots, but can’t go in-game because of a touch-related bug.

Screenshots

Roadmap

Here’s a rough list of major tasks we’re prioritizing at the moment, other than fixing bugs. They will likely be explained in more detail once implemented.

• LLE support for decrypted OS modules

• Shader reversing/recompilation

• Callback system It is common for Vita functions to work with callbacks instead of blocking execution. We don’t have such a system for callbacks yet, so we can’t implement all those functions properly.

• Sync primitive improvements
  • Timeouts
  • More sync primitive implementations

• Variadic function bridging support Some functions from the Vita’s OS modules use functions that accept an arbitrary number of parameters. Currently there is no support for this in our system that bridges Vita (ARMv7) function calls to host (x86-64) calls, so we can’t fully implement those Vita functions.

• Fonts Every homebrew that uses SceFont (a lot of them) can’t boot at the moment, it’s one of the most important modules needed for homebrew to work.

• Dialogs There has been some initial work on this using our new GUI, but more is needed.

• ‘Dynarmic’ integration (replace ‘unicorn’) We currently use unicorn-engine to run ARM code, but we plan to switch to dynarmic at some point (same JIT that is used by Citra and Yuzu), for reasons that will be covered once it happens.

Closing Words

Figuring out things like shaders and shader translation will take a while, but we’re quite excited about solving that and any other challenges we’ll face trying to emulate the Vita. The only limiting factor here is the developers’ spare time, something which our Patreon could help with by funding more time for development.

This post was written by VelocityRa, a developer - unless we get more people willing to help write these reports (or record/edit videos), we can’t promise they’ll be regular and more detailed, so if you’re interested, contact us on our Discord.

Github Statistics

The past three months, 6 contributors pushed 191 commits in 39 Pull Requests, introducing 5,397 added lines and 3,076 deletions in 132 files.

That’s all for now, thanks for reading!

Discuss this post on /r/emulation.