I hope the following explanation will help a bit.
The red distortion will happen *whenever* HEVC-encoded video that uses the BT.709 colorspace is played from an MKV container that has (optional) color information in the header on a newer model Roku (that is, any model that has the video upscaler feature). Unfortunately, this container type (MKV with color info) is exactly the one the Plex server uses whenever it transcodes or Direct Streams. That is what makes the behavior harder to understand.
The problem will not happen if:
- the content is Direct Played from an MP4 container, or
- the content is Direct Played from an MKV container that has had the color info removed, or
- The video is transcoded for any reason. Plex server will transcode the video to burnin subtitles or to reduce bandwidth. But Plex always transcodes video using the H.248 encoder at 1080P or lower. This means the red problem disappears on transcodes, but so does 4K resolution.
The problem will happen if:
- The content is Direct Streamed, even if the original was in an MP4 container. This is because Plex Server uses MKV containers (with color info) when remultiplexing for Direct Streaming.
- The content contains only incompatible audio tracks, such as AAC surround sound. In this case Plex Server will transcode the audio to a compatible format, and then remux the whole into a new MKV container (with a color header), thus triggering the bug. The AAC stutter workaround also triggers the bug by forcing remultiplexing.
If your force Direct Play on an MP4 file that contains incompatible audio, the red problem will disappear, but so will the sound. Ditto for an MKV file with the color info removed. You can't Direct Play what can't be directly played, if you follow my meaning.
In order to workaround the problem, you first have to ensure the file contains audio that is compatible with Roku and your playback environment. Otherwise Plex will transcode the audio and remultiplex into an MKV container. AC3 is the most broadly compatible format. If the original audio is incompatible, you have to pre-transcode the audio track yourself using a tool like ffmpeg. You then have to either mux the content into an MP4 container or use an MKV container and then use a utility like MKVToolNix to remove the color info. Finally, you have to make sure the your player is set up to Direct Play the file.
The workaround, as you see, is pretty complex. That is why so many people struggle with it. You have to get it right, or the red problem persists.