From http://www.gnu.org/licenses/gpl-faq.html#DoesTheGPLAllowMoney : 

Can I release a non-free program that's designed to load a GPL-covered plug-in?

It depends on how the program invokes its plug-ins. For instance, if the program uses only simple fork and exec to invoke and communicate with plug-ins, then the plug-ins are separate programs, so the license of the plug-in makes no requirements about the main program.

If the program dynamically links plug-ins, and they make function calls to each other and share data structures, we believe they form a single program, which must be treated as an extension of both the main program and the plug-ins. In order to use the GPL-covered plug-ins, the main program must be released under the GPL or a GPL-compatible free software license, and that the terms of the GPL must be followed when the main program is distributed for use with these plug-ins.

If the program dynamically links plug-ins, but the communication between them is limited to invoking the `main' function of the plug-in with some options and waiting for it to return, that is a borderline case.

Using shared memory to communicate with complex data structures is pretty much equivalent to dynamic linking.

See also the question I am writing free software that uses a non-free library.