I do not see a problem with the flowchart.
I worry a bit about a "button" that is that large. Even with bluray, you've created a rectangle that covers a lot of the screen. As a test, use a "regular" button that is not near the edges.
Interestingly, it does say untranscoded.
It should not. Are you changing any setting in the H.264 Bluray preset?