Thanks for the suggestion, 3.39 (version code 120017) works indeed. I was going crazy about how in the current version I was able to listen to previously (in older versions of the app) downloaded audio books as well as (new) podcasts, but unable to listen to anything I just downloaded.
Apparently audible has started utilizing widevine in recent versions according to this reddit comment: Reddit - Dive into anything
So while in more up-to-date versions your previously downloaded audio books will work, newer downloads will not be able to play.