So certainly stressed, certainly under a lot of financial strain, among many other types, but this is still a guy who owns a castle, multiple horses, cannon, firearms, suits of armor, and acres of land, and has many henchmen under his control, as well as looser affiliations with peers and nominal subordinates.
The knight errant was absolutely a real thing, and in the earlier era many of the German knights in particular were actually serfs, but by the 15th century most of those families which were still around acquired wealth.
You are probably right about the dregs of society kind of people who were more or less permanently in the role of professional solider. But the main key thing about the later medieval period that distinguishes it from many others, is that most people did not just do one thing. It was very common for petty nobles, gentry, burghers, peasants and members of other estates to spend a season or two fighting, and then go back to their lives. It was very common for example in some of the more warlike towns for young men of journeymen age to spend a season or two on campaign as mercenaries, in order to raise money to buy their way into master status and full citizenship (including earning enough to afford harness and weapons).
Some of those ended up pulled into a semi-permanent warrior lifestyle but outside of some situations like the 100 Years War in France, or some people associated with the Crusading Orders, I don’t really think that was the norm.