Memory Leaks / Problems with Long-Running Symfony / Doctrine Console Applications
Hi!
I recently built several console applications that have to keep running as daemons as part of a more complex website. As soon as I added Doctrine for its highly comfortable ORM functionality, I noticed a significant increase in the applications' memory usage, which made sense because it would load Doctrine's code in order to use it. The worst part of it, however, was that the processes kept eating up more and more memory the longer they ran and with each Doctrine query they executed. Finally the processes ended up being killed by Linux's OOM Killer due to the high amount of memory that they wanted allocated for them.
Clearing the Doctrine Entity Manager ($entityManager->clear()) and triggering PHP's garbage collection manually did not help at all. So I assumed it had something to do with the data that Doctrine accumulates in the background.
During my research I finally stumbled upon this Stack Overflow question: Memory leaks Symfony2 Doctrine2 / exceed memory limit
Apparently, Doctrine uses an SQL Logger to log each and every query it uses when running in debug mode.
Due to the nature of Symfony console applications starting up in the dev environment by default when launching them from the command line, it also kept enabling the debug mode. That in turn enabled the SQL Logger which gathered more and more data and kept it in memory.
I decided to launch the processes by simply manually disabling the debug flag. Using the dev environment is not a problem for me, so I just did the following:
$ php app/console custom:command --no-debug
You could also set the environment variable SYMFONY_DEBUG to 0 before launching the command without the --no-debug flag, as it also respects its value - refer to app/console's source code.
Alternatively, you could start it in a non-dev environment:
$ php app/console custom:command --env=prod
or
$ php app/console custom:command -e prod
or just set the environment variable SYMFONY_ENV before launching it.
This way, the SQL Logger is not enabled.
My processes have been running at stable memory usage size ever since.
Another way would be to deactivate the SQL Logger in the code by principle, but that would be defeating the purpose of the nice built-in features of enabling/disabling the debugging mode, so I rather chose the first path. It might come in handy sooner or later after all, and reverting those changes just to re-enable the SQL Logger would be an unnecessary pain.
I hope this was helpful to you. It certainly took me some time to get behind it.
Viewing Hidden Devices in Windows Device Manager
Hi!
If you are trying to find a device that has been hidden in your Windows Device Manager, for example because you don't have it plugged in at the moment, you might find this little guide handy.
- Open the command prompt ("Start" => "cmd")
- Enter
set devmgr_show_nonpresent_devices=1
- Then start the Device Manager from the command prompt via
devmgmt.msc
- In the Device Manager, click "View" => "Show hidden devices"
I hope this helped 🙂
Thanks for reading!