Using google_cloud_scheduler_job to schedule batch jobs

I'm trying to schedule a batch job using google_cloud_scheduler_job terraform resource.

As per the document , I see only the following options:

• PubSub target
• HTTP target
• AppEngine target

Any suggestions on how to create a batch job scheduler using google_cloud_scheduler_job? Thanks.

1 Answer

Let us split the story into two parts. Let us assume a function ... that when called, will initiate your batch job. You can write this function in a variety of programming languages .. in this example, we'll assume Node. In your Node function, you could (for example) call the DataProc Node.js sumitJob function to instantiate a DataProc job.

Now the question changes from "How do I schedule the execution of my batch job" to "How do I schedule the execution of a function (which executes the batch job)". And here is where the combination of Google Cloud Scheduler and Google Cloud Functions comes into play. Google Cloud Functions allows you to write a code function which is externally triggered by an arriving event. Such an event could be an HTTP request (as WebHook) or a Pub/Sub message. And where can these events come from? The answer is Google Cloud Scheduler. Once you have created your function, you can define that the function be executed/triggered on a schedule. And the result of all of this appears to be your desired request.

A tutorial illustrating Cloud Scheduler and Cloud Functions interacting can be found here.