I recently had to get sendmail working on a machine running a (mostly up to date) version of CentOS with SELinux enabled. After the usual wade through the morass of sendmail configuration options I had something which appeared to be mostly functional. However, it was soon discovered that delivery to programs (i.e., the "|/path/to/mail_handler" kind of delivery) was failing with the unhelpful error message "unknown mailer error 126".
Searching suggested that this error message was supposed to mean that sendmail could not execute the program in question, but most suggestions for the reason involved smrsh, which was not a factor in this installation. Basic system tools could be delivered to (e.g., "|cat >> mymail"), but not shell scripts that included those lines.
It took some time to discover that SELinux was a factor, and that disabling it enabled the delivery to successfully invoke the desired program. Of course, the goal was presumably to keep SELinux enabled but also allow the mail delivery to work, so further investigation was needed.
To make a long story short, the issue turned out to be that sendmail did not have permission to execute user-created files due to them having the wrong security context. In this case the home directories were NFS-mounted and so the handling scripts were all in the nfs_t domain, which sendmail lacked permissions for. The workaround required was to:
- Ensure that the programs being delivered to were all on a local filesystem, so that they did not have their domains overridden to be nfs_t; and
- Explicitly give them a domain that sendmail could execute; in this case I found that bin_t worked.