Introducing Cloud Run Jobs 

@@adigabaie1093 Thank you for the feedback! I will pass on your comment to the product team.

Good to hear you found it useful, Jagdeep, and thank you for the kind words!

Happy to hear you find the videos useful, Santosh!

In my experience as a developer who's used both, these are the major differences: * Cloud Run Jobs let you run a single job for up to 24 hours. With Cloud Functions 2nd gen it's one hour for HTTP functions or 9 minutes for event-driven functions. * Cloud Run Jobs let you configure automatic retries in case of failure. That can't be done with Cloud Functions triggered by HTTP. It can be done for Cloud Functions triggered by other events. * Cloud Run Jobs can start multiple parallel tasks (workers) for you, to finish quicker. With Cloud Functions you'd have to orchestrate parallel work yourself. * Cloud Run Jobs make it easy to run shell scripts. Some tasks (like doing a database backup) are simpler to do with a gcloud command from a shell script, rather than writing code in a regular programming language.

Happy to hear this is useful. Life becomes a little simpler with each eliminated VM.

Unfortunately not; we're looking into adding this in the future.

@@karolinanetolicka5982 thanks for the answer. It would be super helpful for use cases like scheduling re-processing for a specific date or for a specific chunk of data.

@@karolinanetolicka5982 another question: what are the reasonable limits for the number of job definitions? Is it OK to create new job definition for each hourly run? (I'm thinking of baking job run parameters into job definition)

@@elephantum Karolina told me there are no limits on the number of jobs per project - so yes, a new job for each hourly run is reasonable. We do have limits on the number of concurrently running executions in the project and will be introducing garbage collection of old executions. To find the page where limits are documented, search "google cloud run quotas".

Run-specific arguments (the thing you asked for) have now been implemented. Do a web search for "google cloud run override job configuration for a specific execution" and you will find the documentation. Thank you for your feedback last year!

Yes, you can. See the URL that you enter in Cloud Scheduler to schedule a job:cloud.google.com/run/docs/execute/jobs-on-schedule. But be aware that you have to set the authentication header yourself. If you want to trigger your code with an HTTP call, consider putting it inside a regular Cloud Run service instead of a Cloud Run Job.

Good question! Cloud Functions are great if you prefer deploying source code. Cloud Run is great if you prefer deploying containers.

Cloud Functions do not allow you to determine your environment and does not have the ability to run Shell scripts, Cloud Run Jobs can do both.

Pretty sure there's a GH action for it. I saw it a week ago

As you say, it's possible to add regular Cloud Run services with CI/CD via GitHub. Cloud Run Jobs are not there yet, but we will add it.

The Cloud Run job will continue to run until it's done. Make sure to monitor job success by looking at the status of the Cloud Run job, not the status of the Scheduler job - the Scheduler job will show as successful when the Cloud Run job is successfully started, not when it completes.

The Cloud Run jobs API is inspired by the k8s jobs API, but it's not the same. You'll likely need to make a few changes to your yaml file.

Not sure I understand your question. Would you elaborate? If you want to find out more about Airflow on Google Cloud, check out the Cloud Composer documentation.

Not at the moment, but it's something we may add in the future.

@@karolinanetolicka5982 is there any plans to add continuous deployment to cloud run jobs?

@@RodrigoFernandes-fg6me Karolina is out of the office today, so I will respond. Today you can do continuous deployment of jobs with the command line (which means you can do it from a CI/CD pipeline defined in Cloud Build or GitHub Actions) or with Cloud Deploy.

Good question! I don't know of a way to compute the number of workers at startup time. One way of doing it would be to always start X workers, and each worker has the logic to terminate early if there isn't enough work for it. That approach would also decouple the logic in workers from the decision of how many workers to create. In other words, you can change the number of workers without having to worry about updating the worker code, and vice versa.