Memory Leaks / Problems with Long-Running Symfony / Doctrine Console Applications
Hi!
I recently built several console applications that have to keep running as daemons as part of a more complex website. As soon as I added Doctrine for its highly comfortable ORM functionality, I noticed a significant increase in the applications' memory usage, which made sense because it would load Doctrine's code in order to use it. The worst part of it, however, was that the processes kept eating up more and more memory the longer they ran and with each Doctrine query they executed. Finally the processes ended up being killed by Linux's OOM Killer due to the high amount of memory that they wanted allocated for them.
Clearing the Doctrine Entity Manager ($entityManager->clear()) and triggering PHP's garbage collection manually did not help at all. So I assumed it had something to do with the data that Doctrine accumulates in the background.
During my research I finally stumbled upon this Stack Overflow question: Memory leaks Symfony2 Doctrine2 / exceed memory limit
Apparently, Doctrine uses an SQL Logger to log each and every query it uses when running in debug mode.
Due to the nature of Symfony console applications starting up in the dev environment by default when launching them from the command line, it also kept enabling the debug mode. That in turn enabled the SQL Logger which gathered more and more data and kept it in memory.
I decided to launch the processes by simply manually disabling the debug flag. Using the dev environment is not a problem for me, so I just did the following:
$ php app/console custom:command --no-debug
You could also set the environment variable SYMFONY_DEBUG to 0 before launching the command without the --no-debug flag, as it also respects its value - refer to app/console's source code.
Alternatively, you could start it in a non-dev environment:
$ php app/console custom:command --env=prod
or
$ php app/console custom:command -e prod
or just set the environment variable SYMFONY_ENV before launching it.
This way, the SQL Logger is not enabled.
My processes have been running at stable memory usage size ever since.
Another way would be to deactivate the SQL Logger in the code by principle, but that would be defeating the purpose of the nice built-in features of enabling/disabling the debugging mode, so I rather chose the first path. It might come in handy sooner or later after all, and reverting those changes just to re-enable the SQL Logger would be an unnecessary pain.
I hope this was helpful to you. It certainly took me some time to get behind it.
Compiling PHP 5 with IMAP support on SuSE / openSUSE Linux
If you get the following configure error:
configure: error: utf8_mime2text() has old signature, but U8T_CANONICAL is present. This should not happen. Check config.log for additional information.
It's probably because you're missing either the libc-client-devel package or imap-lib and imap-devel. Fire up yast and install those. You should be good to go now 🙂
(I have openSUSE 11.0 and it doesn't have the libc-client-devel package, but I read about it on another page and thought I'd add it, just to be safe 🙂 )