Close. Under this convention, somebody has to do a final test before pushing the merge commit. It can be a simple sanity test, not a full blown integration test. But somewhere somebody at least did a compile check.
If it fails that test, then you push it back to the person doing the work saying so and they need to fix it.
In a GitHub project I've contributed to (dolphin-emu), a compile check for all supported platforms, plus a few tests, shows up next to the big merge button. So the "other party" doing the merge is a robot, and you can still click the button.
I haven't been a committer on any other large GitHub projects, so I'm not sure how common this is.
So, sadly, I don't contribute to really any github projects. This is a new feature to me and does go a long way to addressing my concern. Honestly, it may completely address it. Certainly sounds like it does.
If it fails that test, then you push it back to the person doing the work saying so and they need to fix it.