In one sense it isn't a silly question. There are risks associated with the password manager being compromised.
But there is no logic to what you suggest. If your concern is a keylogger then keeping your passwords in a text file is not going to help you. When it comes to using them you'll either be typing them or copy/pasting, and still vulnerable to the threat you fear. Worse, all the rest of the time you'll be keeping all of your passwords in a single unencrypted file that could be compromised without a keylogger ever having to be installed.
The point is that the password manager gives most people a much more secure, and practical, solution to the problem of having good password hygiene.
You need to mitigate the threat of keyloggers (a vanishingly rare attack in comparison to other risks that you're likely to encounter - whether they be ransomware or phishing sites or whatever) with good security practices. These include a good AV product, good antimalware software, and common sense (which extends to things like running as a limited user account and not with admin rights).
Here's one person's take on it, which seems reasonable. No idea as to credibility: just the first hit on a Google search.