Enhance!

May 25, 2020 12:37 PM

Apparently there is software that will not just upscale a video, but use an algorithm and/or AI to fill in details so what you get has more details than you started with. I guess they're calling it "superresolution" or "AI upscale", if there's a better term let me know. I'd like to try it out - are there good free or FOSS programs that would do this for a video? Bonus points for GUI but I can try CLI if that's what's available.

I searched and tried a couple but couldn't get them to work (either the output was broken or not higher quality than I started with).

I've seen some people recommend using ffmpeg to split a video into images frame by frame and use an image AI upscale/superresolution program to individually improve each frame. I can do that too if that's the best way to accomplish this.

I realize this sort of computerized guessing is not going to be perfect, but I'm willing to try it out and just see how it does.

posted by Tehhund to Technology (6 answers total)

A decade-plus ago when I last looked for something like this I found Focus Magic, for still images. It runs on Windows and Mac and offers a trial version that will process ten images before requiring registration. So using the ffmpeg approach you mention, you could at least get ten frames for free.
posted by XMLicious at 1:00 PM on May 25, 2020

Not free but I see a bunch of projects using this one:

Topaz Video Enhance AI (expensive but has a trial)

AFAIK, before Topaz released their video enhance tool they had a tool just for images ("Gigapixel AI") which is why some recent upscale projects were splitting the videos into images and then batching them through that to get the superior results Topaz seems to provide.
posted by bluecore at 1:32 PM on May 25, 2020 [1 favorite]

Just for the record (and maybe this is already obvious to you), the extra detail that gets added isn't what was actually there, it's more like the statistically most likely detail based on the low-res image. So it's possibly useful if you just want higher-resolution images for aesthetic purposes, but not useful for any kind of detective work or learning additional information from the images.
posted by mekily at 1:33 PM on May 25, 2020 [5 favorites]

My former employer does a lot of super resolution related work in their computer vision department. All their tools are open source and free. Check this out.
posted by MiraK at 1:38 PM on May 25, 2020 [5 favorites]

Best answer: This piqued my interest so I went searching and found 2 that look promising: deep image which gives you the option to pay by image or get a monthly subscription, and let's enhance which has a really similar pricing structure. I'm actually kind of surprised how expensive these things are!

Luckily, software like this is usually based on academic research, and a motivated hobbyist might be able to reconstruct the effect for 'free' (discounting their own labor). These researchers usually share their code on GitHub, here is one example (I have no idea how good it is): EDVR: Video Restoration with Enhanced Deformable
Convolutional Networks. I found that here: https://github.com/topics/video-enhancement, where there are lots of other resources listed as well. Good luck, this sounds like a fun project!
posted by dbx at 1:40 PM on May 25, 2020 [1 favorite]

You might find the DS9UP project interesting--it's an attempt to produce a tutorial/script that an owner of the Deep Space 9 DVDs could follow to upscale them to 4K. From the linked FAQ: "I’m creating a legal route for individuals to upscale a TV show they already own. I will not be creating or distributing any torrents based on my own work. I will be publishing a full tutorial on how to create what I’ve created, once we’ve reached that point."
posted by pullayup at 1:48 PM on May 25, 2020 [1 favorite]