Monday, October 30, 2017

N64 VI Filter

The N64's RDP chip includes a Video Interface (VI) stage that prepares the video for the final output. From the N64 Programming Manual:
The video interface reads the data out of the framebuffer in main memory and generates the composite, S-video, and RGB signals. The video interface also performs the second pass of the antialias algorithm. The video interface works in either NTSC or PAL mode, and can display 15- or 24-bit color pixels, with or without filtering, at both high and low resolutions. The video interface can also scale up a smaller image to fill the screen.
These functions can make a very big impact on the final image of an N64 game, and the ParaLLEl-N64 libretro core exposes the ability to toggle the postprocessing effects of this stage on and off. Turning it off nets you a few frames per second of speed but also gives us a peek behind the VI curtain:
Filtered
Unfiltered
So, you can see that the filter just barely touches the HUD elements but it does some pretty dramatic stuff to the rest of the image. It applies strong antialiasing to the outside edges of objects, which has a big, noticeable effect (so noticeable, you can see it in the thumbnail images) on Mario's hat and the silhouette of the tree, and it does some blurring that smooths out the dithering that is very visible in the unfiltered shot. On actual hardware, the blurring can be toggled off in some games (Quake II, for example, IIRC) or using Gameshark codes. I believe consoles modded with UltraHDMI or etim's N64RGB boards can also switch it off through the boards' firmwares.

Wednesday, October 25, 2017

Using RetroArch via Snappy

I've been working with some folks on trying to get a snap package up and running for RetroArch to go with our FlatPak and AppImage universal linux packages, and it's turned out to be more complicated than I expected to navigate the particulars of the packaging format combined with the restrictions of the security sandboxing.

We announced the package a couple of weeks ago but quickly got reports that users couldn't load files, they were confused as to why the package took a long time to load (only on the first launch, but they didn't know that), and more. Since updating my laptop to Ubuntu 17.10, I decided to "dogfood" the snap package, since that would be the only way I could get in front of the reports and ensure a good experience.

Since RetroArch requires a lot of stuff to look nice and function properly, we use a wrapper script that checks for the existence of that stuff and, if it's not where we expect it, it copies the stuff into the snap's user directory. Since that copying can take a long time, I decided to use notify-send to include some admittedly uninformative notices just to let the user know that nothing is frozen/broken and that we're just copying stuff in the background. The catch here is that you can't use the system's notify-send, you have to include it in the snap as a runtime dependency, under the "stage-packages" in the snapcraft.yaml recipe, like this. I tried adding an icon to make the notifications prettier and more obviously RetroArch-related, but I could never get it to actually see the icon, no matter where I stored it, so I gave up on that.

Ok, so the notifications were a nice little improvement, but we still couldn't actually get to any files to launch them, which makes the program pretty useless. For that, we needed to add the "home" plug to the recipe, like this. This plug is supposed to be auto-connected, so you shouldn't need to do anything to make it accessible to your application once it's in the recipe. However, RetroArch's non-native file-browsing meant that it tried to start in /, which is inaccessible (and in fact, totally invisible) to the snap (and if your snap starts you in an inaccessible directory, you can't ever get out of it), so I added a line to my wrapper that pre-populates the user's retroarch.cfg config file with a line telling it to start in the home directory, where we should have access. I tried using $HOME and ~/, both of which just sent me to the snap's home directory instead of the true home directory with all the files -_-. The solution I found--which is pretty crummy but whatever--is to use a relative path that points to one level above the snap package. That is, ~/../../../

Similarly, I couldn't reach my network shares, which I mount in /media (despite adding any plug that seemed even vaguely related to such a thing to the recipe), so I had to move my mount points into my true home directory and use those same relative paths to everything, e.g. ~/../../../smbshare/Emulation/BIOS for my 'system' directory. Once the mount point is in my true home directory, I could probably put symlinks into the snap package, as well, to avoid the silly relative paths.

The last major issue I ran into was that the *.desktop launcher that shows up when you search for programs in the sidebar kept complaining about not having an "exec" line and then failing to launch because of it. This one was super-confusing because our snap has a *.desktop file (it lives in $SNAP/meta/gui), and that file definitely has an exec line. It turns out that, during installation, snapd generates the *.desktop file that the OS actually looks for and stores it in /var/lib/snapd/desktop/applications. This file is based on the *.desktop included with your program, but if the exec line isn't just like it expects, it will strip it out entirely and not give you any indication of why. Initially, our *.desktop file pointed the exec line to the retroarch.wrapper script that does so much work for us, but snapd didn't like this and rejected it until we switched it to just "Exec=retroarch", which isn't the name of the actual executable but rather the name of the snap itself. It still launched the wrapper script, since that's what our recipe points to, so we're all set.

Since we need to use our script when we launch from a command line, as well, we made sure to end it with $*, but this has a couple of problems that experienced scriptors will spot immediately. First, it's not escaped, so any spaces in file names will break it. Second, it will only accept a single argument, which isn't going to work for us. So, I changed it to "$@" and all is well.

Now, the only issues left that I know of are: 1.) the wrapper script has our nice invader icon, which shows in the sidebar while the script is running, but once it dies off, the icon goes with it and the actual program just has an ugly question-mark/no-icon-found icon in the sidebar and 2.) the snap can't load any dynamic libraries that live outside of its domain, so I can't conveniently compile a test core and then launch from command line to test it with the -L switch. #1 isn't a big deal and #2 probably isn't possible to fix, so it is what it is.

Saturday, October 21, 2017

Running Graphical Programs as Root in Wayland

Fix for Invalid MIT-MAGIC-COOKIE-1 keyCannot open display error when trying to use sudo.

I just updated to Ubuntu 17.04 (and it's a great release; I was on 16.04 LTS before and it's well-worth the update) and noticed that I kept getting the above error when I tried to elevate my privs to edit system files with a graphical text editor (typically gedit, but I've switched to pluma). That's a result of improved security measures in Wayland that we didn't have to worry about with the now-abandoned Unity desktop used in previous releases.

Most of the solutions floating around for this error refer to X sessions (usually X-forwarding over SSH) and don't actually do anything to correct the Wayland issue. However, I came across this solution, which worked a treat:

Step 1.) Create a new local directory to house a custom executable:
mkdir ~/.local/bin/
Step 2.) Next, we'll make a little script that will elevate the privileges for us when invoked (the OP called it 'wsudo', which seems like a good choice to me):
nano ~/.local/bin/wsudo
Step 3.) Paste in these contents:
#!/bin/bash
#small script to enable root access to x-windows systems
xhost +SI:localuser:root
sudo $1
#disable root access after application terminates
xhost -SI:localuser:root
#print access status to allow verification that root access was removed
xhost
Step 4.) Make the script executable:
chmod +x ~/.local/bin/wsudo
Step 5.) Temporarily add our local script directory to our Path for easy access:
export PATH=$PATH:~/.local/bin
Now, you can take it for a spin and make sure it works as expected (by running, for example, wsudo gedit /testfile and making sure it saves okay). If everything is in order...

Step 6.) Permanently add our local script directory to our Path:
echo "export PATH=$PATH:~/.local/bin" >> ~/.bashrc
That's it. Now you should be able to invoke wsudo any time you need to run a GUI program with elevated privs.

Sunday, September 24, 2017

Padhacking a Terrible Genesis 6-button Controller

I recently got a model 1 Sega Genesis and an Everdrive MD and have been playing a lot of the great shmups and arcade ports. The standard 3-button pads are not great, period, but they're especially crummy for those sorts of games (Street Fighter is basically impossible), so I figured I'd seek out some 6-button pads.

Legit 6-button pads from Sega are quite nice, but they're getting more expensive these days (like everything retro, amirite?), so I decided to check out some of the cheap knockoffs. The cheapest ones I could find were going for $8 for 2 pads on eBay and, while I expected them to be shitty, they're worse than I ever imagined:
The Fighting Putt 6B packaging. Both pads I received looked as if they'd been sat upon.
The buttons are so loose I was worried they would fall right out of the cheap plastic casing. The controllers themselves weigh almost nothing and their cord is a measly 3 feet long. The funniest quirk, IMO, is that they only used 4 screws to connect the housing instead of the 5 Sega used, but they put in a fake plastic screw just to keep up appearances:
Very clever, guys. Nobody suspects a thing.
Between the laughably short cord and the awful buttons, I decided to check out the PCB to see if it might be worth putting into an arcade stick (I already have a PS360+ multi-console board, which covers every console I care about except the Genesis/MD, so this would be useful). It turns out that the PCB is actually really great for this purpose, with a common-ground design and nice, big soldering pads for each input:
Here's a shot with wires soldered onto the pads:
And here's one with the solder joints smothered in hot glue for long-term stability:
I hooked it into an existing stick I had lying around and everything works perfectly. After the price of an extension cable, I'm still looking at sub-$10, so not too bad. I wouldn't recommend the Fighting Putt 6Bs for general use, but they're great for padhacking.

Tuesday, August 22, 2017

8bitdo NES30 Arcade Stick Review and Modding Info

I like to play retro games from my couch and I prefer using an arcade stick, but I don't want to have a giant cord stretching across my living room as a tripping hazard. The obvious solution is to get a Bluetooth arcade stick. The only problem: nobody makes them. It seems the people driving the arcade stick market are distrustful of wireless communication due to latency concerns, despite data from a very reputable source suggesting otherwise.

8bitdo had put out a couple of arcade sticks in the past, the FC30 and FC30 Sanwa Edition, but those sticks never got much traction, AFAICT, and they're long-since discontinued now (and I've never seen one come up on eBay). They've revisited the concept recently, though (presumably because their devices are compatible with the Nintendo Switch*, which got a Street Fighter 2 port, and no other company has released an arcade stick for that market), and released their NES30 Arcade stick, which I preordered as soon as I heard about it.

First impressions - Build Quality and Information

The plastic used for the main body of the box feels a little flimsy. It has some flex to it, which isn't encouraging, and there's a lot of empty space inside the stick, though this is actually a good thing when it comes time to start poking around in there. It has a nice, thick, solid metal base with recessed screws and built-in rubber feet, which is a big advantage in my opinion when compared with the flimsy, easily lost rubber feet from the Mad Catz TE and SE sticks (and once the feet were lost, the non-recessed screws would scrape up wooden surfaces and get caught on fabric -_-).

The buttons are knockoff Japanese-style and feel predictably crummy, but passable if you're just going to use it casually. The stick feels pretty decent, really, with none of the gravelly, scraping feelings characteristic of the Mad Catz SE sticks as they slowly ate themselves.

There are 8 full-size (i.e. 30 mm) buttons for A, B, X, Y, R1, L1, R2 and L2 in modern, staggered arcade stick layout, and a smaller button (presumably 24 mm) for Start. There are also smaller non-arcade-style buttons on the control panel for Select, Pair and Turbo. While Select is bindable in gaming software, the Turbo and Pair buttons are not exposed, leaving users with 10 buttons and a 4-way joystick. That is, there is no dedicated "home" button for assigning to "menu_toggle" in RetroArch/MAME.

Wireless connectivity over Bluetooth is quick and painless, and there's no obvious perceptible latency. If you want to play wired and/or charge the stick, 8bitdo has supplied a full-size USB-A-to-A cable, which is, frankly, bizarre. 

Modding

The metal base is held onto the box by 6 small phillips-head screws. Once those are removed, you can pop the base off safely. That is, there is nothing attached to the base that can get yanked out, etc. Once inside, you can see that the wiring is clean and organized, with color-coded wires leading to plastic pin-headers on the board. You can also see the support structure (the hollow tubes surrounding the buttons), which provides a strong backbone where the stick will be seeing the most abuse.
A shot of the insides before I got started on it.
The good news: swapping out the buttons on this stick is a breeze. The stock buttons snap right out and the .110 quick-disconnects transfer over to Sanwa buttons, which are a perfect fit (I swapped in Sanwa 30 mm OBSFS buttons), with no trouble. The stick also has mounting screws that line up perfectly with a Sanwa JLF stick.

The bad news: THIS IS NOT A COMMON-GROUND PCB. That's not a big deal with the buttons (unless you just really like to daisy-chain grounds for tidiness), but it's a very big problem for the stick, since Sanwa and Seimitsu sticks use a common-ground PCB for their switches. In short, this stick is INCOMPATIBLE with Japanese-style sticks without doing some significant modification.

Speaking of the stick, it has a clip-in square restrictor plate/gate and has the control wires soldered directly to Lema microswitches, from Chinese company Zhejiang Lema Electrics Co. Ltd:
Since they were directly soldered, I needed to cut the wires, making this the first destructive modification so far.

The Lema switches are pretty close in size and shape to the tough-as-nails Cherry microswitches you would find in Happ/IL sticks and buttons, and I decided to swap them out for some I had in an old Happ Competition joystick.
The result is satisfyingly clicky and extremely light (that is, there's barely enough resistance to bring the stick back to center; some people will despise this). I was able to pull off 360/720-degree motions easily and reliably, but I'm not 100% convinced that I want to stick with this setup permanently, so I used insulated alligator clips rather than soldering .187 quick-disconnects to the wires in case I decide to swap it out with other switches in the future.
The extra-roomy case came in handy here for holding my insulated alligator clips
The restrictor plate/gate is held in by 4 little screws and 4 clips. Once the screws are out, the clips are nice and easy to manipulate, unlike the ones on JLF sticks, which are notoriously difficult to work with. I didn't check to see whether Sanwa plates would snap in, but it looks pretty likely. I might swap in an octo-gate at some point and will update this post if I run into any issues. [Update 10/27/2017: by request, I swapped in the Sanwa octo-gate from one of my other sticks and it fit just fine. The Sanwa plate is thicker than the stock plate, so I really had to cram it to get the clips to snap into place, but otherwise, it's no problem.] Here you can see the Cherry microswitches fit in nice and snug under the stock plate:
I didn't get around to testing it, but I suspect the longer, screw-down Happ/IL American-style buttons would fit just fine in the case, since it seems to be a little taller than the Mad Catz SE boxes, which were only about a quarter of an inch too short to fit them comfortably. [Update 10/27/2017: I was wrong, they're almost exactly the same height as the SE box, so the long-stem Happ/IL buttons won't fit, but the short-stem ILs will.]

*Note, the wired vs wireless issue seems to actually be in favor of wireless on the Switch, oddly enough: https://www.youtube.com/watch?v=avvmck40cIw

Friday, July 7, 2017

RetroArch shaders ported to ReShade

I've frequently gotten requests to port RetroArch's library of shaders to ReShade's format for use with other programs/games, but I'm a Linux guy for the most part, so I never had the inclination to do such a thing. Thankfully, ReShade user Matsilagi is/was so inclined and he ported a bunch of them and made them available in this github repo:

I haven't tested all of them, but the ones I've seen look perfect, so they should be ready to go with all of the low-fi CRT/NTSC/PAL goodness.
Artifact Colors
CRT-Geom and CRT-Lottes
PAL
NTSC
R57-PAL
3DFX
GTU-v50
EGA Filter


Thursday, June 22, 2017

RetroArch Tone-mapping LUT Shader

Reshade has long had a shader, LUT.fx, that enables users/designers to do tonemapping and other color adjustments without touching any code. Instead, they can do all of their adjustments in an image editing program, such as Photoshop or GIMP, which many people are familiar with already. While my image-adjustment and color-mangler shaders can be used to accomplish those same tasks (Pokefan531 did a great job modifying them to produce his handheld color shaders, after all), they're awkward to work with, since an artist has to make all of his/her changes by twiddling esoteric values in the shader settings menu.

So, I decided to port the Reshade shader to RetroArch so our users could get in on the fun. I ran into some unexpected behavior with the direct port, though, and decided to adapt another similar shader instead. This one ended up having the same weird issue, which I think is related to undefined behavior, but I put a stupid workaround in the shader to mostly deal with it.

Anyway, here's how you use it:

First, take a screenshot that you want to use as your reference (I'm going to use the Sonic the Hedgehog title screen) and then take one of the passthrough palette textures that come with the shader (they're the png files located in reshade/shaders/LUT, named for their color depth). Then, in Photoshop or GIMP or Paint.NET or whatever, open your reference screenshot and paste the palette image down below it:
I find the easiest way to do this is to go to the image menu > canvas size and then anchor it from the top and increase the canvas Y-axis measurement by the height of the palette texture (in my example, I'm using the '16' texture, so I made the image 16 px taller). Paste the palette image in there, move it to the bottom-left corner and then merge the palette layer with the screenshot layer.

Next, do whatever it is you need to do to make the picture look like you want it. That includes brightness/contrast, hue/saturation, indexed color, different lossy colorspace conversions (such as by switching to CMYK colorspace and then back to RGB). I'm going to do a simple hue shift in my example because it's easy to spot:
Once you have it all set, we need to isolate the palette from the screenshot. I think the easiest way to do this is to go back to image > canvas size, anchor from the bottom left and enter the size of the palette (the width of the passthrough palettes is the height squared; I used the 16-bit palette, so it's 16 * 16 = 256 px). Save that image under a new name (I called mine 'hedgehog-palette.png'; if you're using a different palette depth from the default 16, it's probably a good idea to put that into the filename somewhere so you don't forget it) and then drop it into your reshade/shaders/LUT directory with the other palette images.

Now, open the LUT shader preset (cgp/glslp/slangp file) in a text editor and change the line that points to the palette (probably line 7, but YMMV):
SamplerLUT = shaders/LUT/16.png
becomes
SamplerLUT = shaders/LUT/hedgehog-palette.png
Save and exit and then fire up RetroArch, load a core and some content and then load up the shader. It should apply those same color transformations to the game image, like so:
If you used a different bit-depth palette from the default 16, your colors may look crazy and messed up at first, in which case you need to go back into the quick menu > preview shader changes and then change the "LUT Size" runtime parameter to match.

This shader is available from the online updater and/or git in GLSL, Cg and slang shader formats.

Analytics Tracking Footer