x264 Encoding Guide - Fenrirthviti/obs-studio GitHub Wiki

Preamble

So, you want to learn more about video encoding? First, I would highly recommend checking out this fantastic video by Tom Scott: Why Snow and Confetti Ruin YouTube Video Quality. He does a great job of giving a quick primer on how video encoding works, and you will hopefully have a better understanding of the topics we'll be going over.

First, let's explain what this guide is not. This is not intended to be a fully detailed technical explanation of how x264 works; there are far better guides out there than what I can provide here. If you're interested in the nitty-gritty, head over to the doom9 forums, FFmpeg docs, or the x264 website and start digging. This is also not intended to be a "best settings" guide, and I will not recommend any specific settings. This is intended to help you understand how video encoding in general works, and how to better identify potential issues with your settings and help you know where to look to correct them.

We will be focusing mainly on streaming, as a vast majority of OBS users will be using the x264 encoder to stream. For local recordings, your choice of encoder is far less relevant than your actual settings and in many cases a hardware encoder will be better suited for you. You can learn more about local recording settings in this guide here: http://obsproject.com/forum/resources/obs-studio-high-quality-recording-and-multiple-audio-tracks.221/

The very first thing that is important to understand is that video encoding is a very resource intensive process, especially when attempting to do so in real-time. Hardware encoders - such as Nvidia NVENC, Intel QuickSync, or AMD VCE - can help with this as they are dedicated to the task, so they can process quickly. As a trade-off the overall quality per bitrate is lower than the CPU-based x264 in nearly all cases. For streaming where bitrate is usually a constraining factor, x264 is the current best option for getting the most quality out of your stream.

It has only been in recent years that your standard consumer-grade computers have reached the point where they can realistically provide the processing power to do live video encoding. Keep all this in mind when you wonder why your 8 year old dual core Pentium CPU cannot encode 1080p 60fps without failing miserably. Even the most powerful consumer CPUs can still struggle with the load of encoding a high-resolution, high-fps stream.

There are two primary components to the x264 encoder we'll be looking at: presets and bitrate.

Presets

x264 has several CPU presets, in increasing order from low CPU usage to high CPU usage: ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo.

By "preset", it means exactly what it sounds like: a set of pre-determined settings for x264 so that you don't have to set them all manually yourself to tweak things. These sets of settings have been tested by lots of people and are great for general use, depending on what you want to get out of your encoder. The actual details of what the settings are can be found here: http://dev.beandog.org/x264_preset_reference.html

The basic idea is that, all things being equal (same bitrate, etc), less CPU usage would result in worse quality, and more CPU usage would result in better quality, because the presets change how much time the encoder spends compressing each frame to look good within its setting constraints. Sometimes you need to reduce your CPU usage in order to get good performance, and the higher CPU usage presets can be difficult to use effectively with average consumer CPUs.

The last thing to note is that any preset lower than medium will have significant diminishing returns, and is not really worth the extra CPU cycles for streaming scenarios. Unless you are squinting at two identical streams side by side, you will not notice a difference. That said, if your CPU can handle it, there's no reason (outside your power bills) not to use them.

Here we have put together some comparison examples for how this actually looks in practice.

All these test are performed with exactly the same source video and bitrate, only the preset has changed.

Bitrate

The amount of energy the CPU spends compressing each frame isn't the only factor in video quality. Bitrate is also important, as it determines how much information you can put into each frame of video. If you are allowed to cram more data into each frame, you don't need lots of CPU spent on compression, so you can make each frame look better just by cranking up the bitrate. Remember how the Tom Scott video looked when he simulated lowering bitrate, with all other settings left the same? The same is true the other way. If you increase bitrate, you can make the video look better quality.

Thus, you can get a good-looking video with relatively low CPU usage by using a low-CPU usage preset (like superfast) with a higher bitrate. Just note that the amount of bitrate you'll need for this can vary greatly depending on the resolution and FPS you are trying to stream at. A 1080p 60fps stream at only 4000kbps bitrate using the ultrafast preset is not going to look very good. For reference, the YouTube encoding settings list is a great place to start. The list below differs slightly, and would be my personal recommendation as a starting point.

Resolution Bitrate FPS
853x480 800 - 1200 kbps 30
1024x576 1000 - 3000 kbps 30
1280x720 3000 - 5000 kbps 30
1920x1080 5000 - 8000 kbps 30
2560x1440 8000 - 12000 kbps 30
3840x2160 12000 - 20000 kbps 30

They are assuming x264 encoder with the veryfast preset, and low to medium motion in your scene. For 1080p 60fps in a high-motion scenario (like an action or FPS game), you would likely need more than 8,000kbps of bitrate at veryfast for it to look smooth during playback.

The end result of using a lower encoder preset and upping the bitrate will probably look a bit different then comparable bitrate at higher presets, but the goal is to get roughly the same quality by trading off CPU usage for bandwidth. I recommend trying both and see which one works for you in terms of quality, CPU usage, and what your connection and streaming service can handle.

For comparison, here is the same scene encoded using x264 veryfast in both a low-motion high-detail scene, and a high-motion scene.

As before with the preset test, these tests were performed with the exact same source video and preset, only the bitrate has changed.

I hope that this has helped you gain a better understanding of both how video encoding works, the importance of bitrate, and the overall impact of changing these settings in OBS. As always, the road to finding the perfect settings for your stream is to test, test, and test again. If you have any further questions, our forums and support chat are always open.

Happy creating!