I use Horizon, it's great! I process a lot of data with it, currently sitting at 1800 jobs/minute. I made a big NOOB error when I started by passing the data to be processed into the job itself, which was occasionally multi-megabytes. This worked fine most of the time but if anything went wrong, the multiple job attempts plus storage of errors meant it quickly filled up the memory and made the problem catastrophic! It also made the horizon dashboard sluggish and fail, as it tried to show me the multi-megabyte data in each job attempt. The solution: Store the data separately, either on disk, DB or in Redis, and pass only an ID or reference to the data into the job. Then Horizon will fly.
Great comment, working at a similar scale and found this myself. In my case nested eager-loaded relationships were the usual explanation. Just pass a model ID to the job, then load the model from the DB in handle(). This also ensures you’re using the most up-to-date model instance at runtime too!
@@vasiovasio it depends what the jobs are doing. If it’s something simple like sending an email, a basic VPS in the region of $50/mo. could handle this with 10 Horizon workers. 1800 jobs/min. across 10 workers = 3 jobs/sec./process. More complex tasks will vary significantly.
@@rcnhsuailsnyfiue2 Thank you! In these days of Unreal and I mean Unbelievable Inflation ( after Covid and the start of the war in Ukraine, the food pricing in my country, Bulgaria are basically Three times higher than 2019 ) the prices of hosting, servers, cloud, storage and everything in the sphere is relatively normal as a price and I'm thankful about it, because tell me for other opportunity to test some business idea for $50-$100 per month and everything else is your work, time, energy and dedication!
These are awesome video's to help people get to see some of the cool sides of modern PHP development options. I have been using Laravel since version 3 and the whole ecosystem has come so far. Thank you for making this series.
Why there is no comprehensive and up to date laravel course that shows how to build an API? We don't need intertia, fullstack, htmlx, blade, just API, like in 99% of companies.
Great intro video! I recently discovered Horizon and now using it at a reasonably big scale. Just one question. If I don't specify the queue worker name while dispatching a job, how do I make sure that the processes get distributed to different queue workers automatically? Currently, all of them are going to the default worker only.
Gonna be honest, don't really care for Laravel. But anything that Aaron Francis makes deserve consideration. Probably one of the best "dev evangelist"-type channels.
I have used Laravel Horizon for years and it's even been a deciding factor for bringing Laravel into a project. That said, I still don't get why the environment setup is so overy complicated and hard-coded into to config/horizon.php file. If I have 3 or 4 environments and need to scale up or down the worker limits on one of them then I would rather just update the .env file for that server than push a commit with per-server/environment logic...
@@rcnhsuailsnyfiue2 yes that’s my preference but that’s not the documented /recommended way to do it. Docs show a big array of repeated hardcoded values per environment, rather than simply setting your own values per queue in the .env file which seems more logical to me
@@JonnyJKFthe hard coded configs are just fallbacks/defaults and an example of what can be configured. The env() calls should be telling enough to hint at the fact, that they are meant to be overridden by the actual environment variables in your systems.
Spin up a copy of your app on another machine in the same data center/hosting region. No need to handle frontend requests, this copy only needs access to your central Redis database, and therefore can process queued jobs like any other copy of your application. Horizon will automatically assign jobs to workers, both multiple workers on one machine, or across workers on multiple machines. Horizontal scaling is simply a case of spinning up more instances of your app.
Laravel Horizon. The point at which you cross over to coding PHP with no hope of returning. You become increasingly redshifted until you simply vanish from sight.
