Decrypt a PKCS#1 v1.5 ciphertext.
Attention:
You should never let the party who submitted the ciphertext know that
this function returned the sentinel value.
Armed with such knowledge (for a fair amount of carefully crafted but invalid ciphertexts),
an attacker is able to recontruct the plaintext of any other encryption that were carried out
with the same RSA public key (see Bleichenbacher's attack).
In general, it should not be possible for the other party to distinguish
whether processing at the server side failed because the value returned
was a sentinel as opposed to a random, invalid message.
In fact, the second option is not that unlikely: encryption done according to PKCS#1 v1.5
embeds no good integrity check. There is roughly one chance
in 2^16 for a random ciphertext to be returned as a valid message
(although random looking).
It is therefore advisabled to:
- Select as sentinel a value that resembles a plausable random, invalid message.
- Not report back an error as soon as you detect a sentinel value.
Put differently, you should not explicitly check if the returned value is the sentinel or not.
- Cover all possible errors with a single, generic error indicator.
- Embed into the definition of message (at the protocol level) a digest (e.g. SHA-1).
It is recommended for it to be the rightmost part message.
- Where possible, monitor the number of errors due to ciphertexts originating from the same party,
and slow down the rate of the requests from such party (or even blacklist it altogether).
If you are designing a new protocol, consider using the more robust PKCS#1 OAEP.