To enhance the security of your Outlook application as you described, you possess a couple of options. Firstly, consider setting up Windows Hello, which allows you to use a PIN, facial recognition, or a fingerprint to unlock your device and applications like Outlook. This adds an extra layer of security every time you access the app.
For using a Yubikey, you would need to integrate it with your Microsoft account for two-factor authentication (2FA). Although directly configuring Outlook to require the Yubikey each time it opens isn't supported natively, having 2FA enabled on your Microsoft account adds that extra security layer by requiring the key whenever logging in from an unrecognized device or location.
Make sure also to keep your software updated consistently for optimum security and check if there are any specific add-ons or plugins for Outlook that could further secure access.