I suspect that you're right, and that this is a large part of the problem.
There are two variations of the phantom manager. One is the person who, as you say, year in and year out does not make trades, does not submit draft lists, does not play games. In several of my leagues, I wind up e-mailing the commish to have him go in and move players up or down to deal with over-usage.
The second though is the more aggravating, and that's the person who takes the year off. I was gone for 4 weeks this summer, and it was very difficult to manage my rosters, let alone play the games. And yes, people get sick, move, change jobs, have kids, etc. But, I suspect that there are a fair number of people who just drop out for a year (although they keep their teams). Maybe they don't have a good team, maybe they just want to do other things. This is the owner who you've been able to deal with in the past, but suddenly you get no replies, either in e-mails or in submitted trade offers.