I average about two full length (400-500 pages, give or take) books per week, if my schedule allows. I rarely have more than 3 or 4 going at one time, and so far I haven't had any issues like what you describe. However, it depends a great deal of what you are reading, rather than just the number. For example, the "A Song of Ice and Fire" volumes by George R.R. Martin have literally hundreds of names to keep track of, even if you discount the nicknames, first name/surname mash-ups, and family symbols. If I actually tried to keep track of three or even two books like that at once, I probably couldn't do it. But I read almost purely for entertainment, so basically I just ignore all that and can still enjoy the story, and over time (the set is several books and the shortest so far is like 800 pages) you really get to know the important characters anyway, without exerting the effort.
It is well known that humans are quite lousy at multitasking, and people consistently overestimate their abilities to manage it...honestly if you are reading two books at once, you're doing just fine, and you'll probably retain the information better anyway. Good to know there are still avid readers out there!