ESP32 Mailbox works on USB power but not battery

The article said that they were getting 19ma in sleep mode with the board they were using and this is very similar to your results.
Removing the LED from your board might save 1-2ma, but I think the main loss is in the voltage regulator circuit.
Our friend with the Swiss accent has a few videos on the deep sleep suitability of a variety of boards.

Pete.