First and foremost, as it currently stands today both Flight Simulator and Prepar 3D are all 32bit programs and in this 32bit world architecture the total memory the simulator can access and use is limited to 4gb max, and Please use a 64bit OS for full benefit.. Regardless of how many GB of graphics or PC memory you have the simulator can only use 4gb, so forget about rushing out to throw a new graphics card or memory at the problem in hopes of fixing the OOM issue - It simply will not work, So, what to do? Below is my basic strategy to manage the limitation better. For the remainder of this article we will refer to OOM as Virtual Address Space (VAS)
In my early days, I would accept and automatically run those extra bits (dll) from each module installed, unknowingly chewing up my limited VAS - even when I was not using the plug in for that particular flight. My new approach is to explicitly allow or deny each dll as I am prompted. This provides me the ability to not run any add-ons that are not associated with my intentions for that particular flight.
As each module is being loaded I can deny or allow it to be loaded + add to trust list if it is necessary for FSX overall - in this case Yes.
So now rather than "allow" and "trust" everything I install, I choose to individually allow or deny depending on my flight needs, this saves valuable VAS for more important things like working the cockpit of that Boeing 777.
For Example: The REX Essential Plus Overdrive has a weather add on and adds a MV_WXM.dll entry in your dll.xml file and each and every time you run Flight Simulator the module attempts to run as well - which is fine if your wanting to run weather systems , since I already have Active Sky weather, I simply do not need to chew up VAS with a module I am not using.
Many of you have already figured this strategy out and may have created special fsx.cfg and dll.xml files for special purpose flying. I basically have 4 sets I use - If I am going for a longer flight in my PMDG 777 then I use my special fsx.cfg and dll.xml set that is customized for that one particular aircraft, this give me the best chance of avoiding VAS out of memory during flight.
Here the Flight has been fully loaded into Flight Simulator - notice the VAS shows FSX using 694mb - a good start!
The second biggest thing to consider is scenery addons such as highly detailed airports - those eat up VAS faster than a rabbit. So, you will need to experiment a bit - ask yourself what is more important. Once the flight environment is loaded and your sitting at the gates, use a tool such as Process Explorer to measure how much VAS is being used - if your at 3.8gb I highly doubt you will finish the flight without a simulator crash.
Making the decision on what your objective is before running Flight Simulator. and not loading those modules your not going to use, goes a long way in avoiding the OOM,
Here are a couple Tools I use to help me better manage VAS.
Process Explorer (just google it)
Scenery Config Editor (Google it) please note it uses Java.