You can get over 400 lm/W in theory with RGB but right now red and green LEDs lag blue in terms of efficiency. Also, that 400 lm/W number assumes 100% efficient emitters. In reality after two decades of development we're around 75% wall plug efficiency for blue emitters. Most likely efficiency will top out at 80% to 85%. If you reach similar numbers for red and green emitters then maybe you'll be able to emit white light at ~320 to 340 lm/W. That's a big if. Also note that this is for CRI 80 light. The numbers drop for higher CRIs.
Looking at blue plus phosphor LEDs, you'll always have losses in the phosphor conversion process. For CRI 80 light the efficacy of the emitted spectrum for phosphor white LEDs is roughly 330 lm/W. For CRI 95 it's about 270 lm/W. But you lose perhaps 15% in the conversion process. So even with 100% efficient blue emitters the best we can do with CRI 80 white is around 280 lm/W, maybe close to 300 lm/W with better phosphors. For CRI 95 the maximum is around 230 lm/W, assuming a 100% efficient blue emitter. As I said, we'll never get there. If we get to 85% efficient blue emitters then you'll have CRI 80 LEDs with roughly 240 to 255 lm/W efficiency. Note how close the best production LEDs already are to those numbers. With CRI 95 you'll be lucky to break 200 lm/W. Still, these numbers are more than double that of the next best light source.
We don't really "need" to go much further in terms of efficiency improvements anyway. From my standpoint, the biggest benefit of slightly more efficient LEDs is better heat management, not slightly less energy use. For example, those 230 lm/W LEDs I linked to earlier have a wall-plug efficiency of perhaps 70%. If we're using them in a 200 watt light bulb replacement (3200 lumens), then we'll be using 13.9 watts of power and generating 4.2 watts of waste heat. If we manage to reach a wall-plug efficiency of 80%, then we'll be using 12.2 watts of power but generating only 2.5 watts of waste heat. A relatively small improvement in wall plug efficiency cuts waste heat generation by 40%, even though you're only using 12% less power.