Automating the ingestion and processing of data

Overview

Graft allows you to iterate on your workflow by importing data, training and applying models. After you are satisfied with the results, you will want to automate the workflow to get updated results on a regular basis and to continue to improve your model(s). Scheduled workflow allows you to run a workflow automatically on a given cadence that you specify.

App users

By default, when you click the icon in an App TRY IT screen we automatically schedule the ingest and processing of your data on a daily basis, if you need a different schedule you can follow these steps.

Scheduling capabilities are found in the Jobs tab within the Entity Dashboard

On the left hand menu click on Data
and then select the Entities tile
From the right hand side bar select the entity you wish to set up a schedule For App users you can see your Entity name in the right hand table
You will be presented with the Entity Summary dashboard for your selected Entity. Click on JOBS to see the current job list related to your Entity or App

Job Types & Precedence

All jobs are listed in operating sequence but not all can be scheduled. You will also see independent jobs for different embeddings. E.g. if you have created multiple configurations for an app there will be jobs related to each configuration.

The jobs that can be scheduled will have the button, once schedule will be displayed.

	Ingest
	Apply Trunk model (embed)
	Apply Prediction

When you schedule a job all the prerequisite jobs will be scheduled. For example to be able to Apply a trunk model the ingest job must have run and to Apply a prediction requires that Ingest and Apply Trunk model run. While the prerequisites will not show the button they will run as needed. You can also schedule these jobs if required for other purposes.

Scheduling

To Schedule a job click on the button in the jobs table.
Clicking on the check box will activate the schedule settings, type in a value or use the counter buttons (appear on mouse over) at the end of the number input box. Click on the icon to display the time options and select the most appropriate for your application.
To accept the settings click OK, to cancel and return to the job list click CANCEL.
Once scheduled will be displayed in the job's row The scheduled job will immediately run (along with associated prerequisites) and recurr based on the assigned schedule.

Checking a Job Schedule

Any Job which has the button can be moused over and the current settings will be displayed

INCOMPLETE WORKFLOWS

If the processing of a workflow is still running when the workflow is scheduled to run again the new workflow will only run those jobs which were complete from the previous run. The "In process" job will not be run. On the subsequent workflow run all jobs would be run, provided there are none running, catching up on all the processing. If you would rather not wait til the next scheduled workflow you may manually run the job from the jobs tab or from the entity dashboard. For example should an hourly workflow to ingest, apply trunk model and enrich still be running (enrichment has not completed), the next workflow will only ingest and apply the trunk model to the new data