Automated Recipes
Deploy and operate automated recipes through the v3.1 API.
An automated recipe turns a template into a SQL view in your workspace destination, built from the extractions the template needs.
Deploying a recipe takes four calls, in this order:
GET /api/v3.1/automated-recipe-templates/- pick a template.POST /api/v3.1/automated-recipes/- create the recipe. It is created empty.PUT /api/v3.1/automated-recipes/{recipe_id}/- supply data sources, connections and accounts. This step is mandatory.POST /api/v3.1/automated-recipes/{recipe_id}/build/- start the build, then pollGET /api/v3.1/automated-recipes/{recipe_id}/untilstateisready_to_workorfailed.
Every path ends with a slash; a 301 means the slash is missing from your URL.
With a bearer token the workspace is the one the token was issued for; with cookie
authentication, send the workspace id in the X-IM-WORKSPACE-ID header.
All errors use one envelope - see Errors.
See the Deploying recipes guide for the full flow.
List recipe templates
Lists the recipe templates your agency can deploy through the API, in catalogue order. A template belongs to the agency, not to a workspace, so the same id can be deployed in every workspace of the agency.
Only templates that can be deployed through the API are returned, so this list can be shorter than the catalogue the interface shows. Templates are created and edited in the interface only; this API reads them.
Pick a template here, then create a recipe from its id. The page holds 20 templates by default and 100 at most; next and previous are full v3.1 URLs.
query Parameters
searchCase-insensitive substring of the template title. An empty value is rejected with 400 invalid_query_parameter.
data_sourcesComma-separated data source names, for example facebook,google_ads. A template is returned when it supports at least one of them. An unknown name, an empty value or a bare comma is rejected with 400 invalid_query_parameter.
pagePage number, starting at 1. A page beyond the last one is rejected with 400 invalid_query_parameter, not answered with an empty list.
page_sizeTemplates per page, from 1 to 100. A larger value is rejected with 400 invalid_query_parameter; it is not silently capped.
List recipe templates › Responses
One page of recipe templates
countTotal number of templates matching the request.
nextFull URL of the next page, or null on the last page.
previousFull URL of the previous page, or null on the first page.
Templates on this page.
Get a recipe template
Returns one recipe template with the templates it is built on and the columns of the table a recipe from it produces. Use it to check a template before you deploy it: output_columns is available without deploying anything, and an empty output_columns means the schema could not be read, not that the table has no columns.
template_id is an integer. A value that is not a positive integer answers 404 not_found, the same as a template that does not exist, belongs to another agency or cannot be deployed through the API.
path Parameters
template_idRecipe template identifier, as returned by the template list.
Get a recipe template › Responses
The recipe template
idTemplate identifier. Pass it as template_id when you create a recipe.
titleTemplate name. It can be changed in the interface, and the name in force when a recipe is created decides the name of that recipe's output table.
descriptionTemplate description, as entered in the interface. Can be empty.
Data sources this template supports. The data_sources you send when you configure a recipe must be taken from this list.
Categories the template is filed under in the interface, in display order. Empty when the template has none.
originWho the template comes from. improvado - a template we provide; agency - a template your agency created by promoting one of its own recipes. Agency templates are the ones the interface shows as custom. New values can be added, so tolerate unknown ones.
build_modeHow a recipe from this template builds when one data source delivers no data. progressive - that data source is left out, the rest is built and the recipe becomes ready_to_work with degraded: true; if every data source fails, the recipe fails. atomic - a single failing data source fails the whole recipe. In both modes individual accounts that did not deliver never fail the recipe on their own. New values can be added, so tolerate unknown ones.
levelDepth of this template in its own tree; 0 for a template you can deploy.
Templates this one is built on, nested to any depth. A recipe created from this template builds every node of the tree.
dependencies_countHow many templates this one depends on, counted over the whole tree.
Columns of the table a recipe from this template produces, available before you deploy anything. An empty list means the schema could not be read - treat it as unknown, not as "no columns".
created_atWhen the template was created.
updated_atWhen the template was last changed.
List recipes
Lists the automated recipes of the workspace the request is authenticated for, the most recently changed first.
Every field you need to triage a fleet is already here: state, degraded, not_delivered_reasons and error. Read the detail of a recipe only when you need its configuration or the per-data-source breakdown.
The page holds 20 recipes by default and 100 at most; next and previous are full v3.1 URLs.
query Parameters
template_idReturn only the recipes deployed from this template. A value that is not a positive integer is rejected with 400 invalid_query_parameter.
stateReturn only the recipes in this state. The value has to match one of the four snake_case states exactly; anything else, including a v3 value such as Ready to work, is rejected with 400 invalid_query_parameter and the allowed values in details.allowed_values.
pagePage number, starting at 1. A page beyond the last one is rejected with 400 invalid_query_parameter, not answered with an empty list.
page_sizeRecipes per page, from 1 to 100. A larger value is rejected with 400 invalid_query_parameter; it is not silently capped.
List recipes › Responses
One page of recipes
countTotal number of recipes matching the request.
nextFull URL of the next page, or null on the last page.
previousFull URL of the previous page, or null on the first page.
Recipes on this page.
Create a recipe
Creates a recipe from a template in the workspace the request is authenticated for. The recipe is created empty - no data sources, no connections, no accounts - so the next call has to be PUT /api/v3.1/automated-recipes/{recipe_id}/. Starting a build before that answers 409 recipe_not_configured.
One recipe per template per workspace. A second create from the same template answers 409 recipe_already_exists and names the recipes that already exist. Do not create recipes from one template in one workspace in parallel: the second call loses the race and gets the same answer.
title names the recipe and nothing else. It is optional and at most 255 characters; if it is missing, null, empty or only spaces, the template supplies the default name. The name of the output table is derived from the title of the template at the moment the recipe is created, never from title in this body, and the recipe keeps that name for good: renaming the template later changes the table name only for recipes deployed after the rename. To read the physical name, take datatable_id from this response and call GET /api/v3/data-tables/{data_table_id}, then read sql_name.
Create a recipe › Request Body
template_idIdentifier of the template to deploy, as returned by the template list. It has to belong to your agency and be deployable through the API; anything else answers 400 template_not_found.
titleName of the recipe, at most 255 characters. Omit it, or send null, an empty string or only spaces, to let the template name the recipe. It has no effect on the name of the output table.
Create a recipe › Responses
The recipe was created. It holds no configuration yet, so data_sources, connections and sources are empty and last_build_at is null.
recipe_idRecipe identifier. A string, even though it is made of digits.
template_idThe template this recipe was created from.
template_titleCurrent name of that template.
titleRecipe name. When you create a recipe without a title, the template supplies the default.
workspace_idThe workspace this recipe lives in.
stateWhere the recipe stands. extracting_data - data is being collected; building_dependencies - the tables are being built; ready_to_work - the last build finished and the output table can be queried; failed - the last build failed, see error. ready_to_work means built, not built completely: it is built from the extractions that delivered, and how complete that is shown by degraded and sources. New values can be added, so tolerate unknown ones.
degradedWhether the last build was incomplete. Only meaningful while state is ready_to_work: in every other state, a failed recipe and a running build included, it is false. true - the build left something out, either a whole data source or some extractions; see not_delivered_reasons and sources. false - everything in the build delivered. null - the completeness of the last build is unknown because that build ran before version 3.1.
not_delivered_reasonsThe distinct reasons behind an incomplete last build, sorted and deduplicated across all data sources. Empty when degraded is false or null.
errorWhy a build failed. On a recipe this is not null if and only if state is failed; on an entry of sources it is not null if and only if that entry's status is failed. Build error codes are a namespace of their own, separate from the codes of HTTP error responses - a few names appear in both. The catalogue is on the Errors page.
datatable_idThe recipe's output data table, or null while none is registered. Resolve the physical table name with GET /api/v3/data-tables/{data_table_id} instead of deriving it from a title.
last_build_atWhen the last build of this recipe was started, no matter what started it. null means the recipe has never been built. For recipes last built before version 3.1 this is an approximation: the time the recipe record last changed, which is often the end of that build rather than its start.
created_atWhen the recipe was created.
updated_atWhen the recipe itself last changed. It says nothing about builds - for those use last_build_at and state.
data_sourcesData source names the recipe is configured for, sorted. Empty on a recipe that has been created but not configured yet.
allowed_data_sourcesData source names the template allows, sorted. Everything in data_sources has to come from this list.
The connections and accounts the recipe reads. This is the full set the recipe holds; configuring a recipe always sends the full set, and nothing can be taken away from it.
sync_from_dateThe date the recipe collects data from. Always set in a response, even though it may be omitted from a configuration request.
What the last build did with each configured data source. One entry per name in data_sources.
Get a recipe
Returns one recipe with the configuration it holds and what the last build did with it. This is the operation to poll after starting a build: repeat it until state is ready_to_work or failed.
ready_to_work means built, not built completely. A build runs on the extractions that delivered, so check degraded and sources as well: degraded: true says something was left out, not_delivered_reasons says why in one line, and sources gives the breakdown per data source. error is filled in if and only if state is failed.
A recipe that was created but never configured comes back with an empty data_sources, last_build_at: null and state: extracting_data. Configure it before you build it.
path Parameters
recipe_idRecipe identifier, as returned when the recipe was created. A string of digits.
Get a recipe › Responses
The recipe
recipe_idRecipe identifier. A string, even though it is made of digits.
template_idThe template this recipe was created from.
template_titleCurrent name of that template.
titleRecipe name. When you create a recipe without a title, the template supplies the default.
workspace_idThe workspace this recipe lives in.
stateWhere the recipe stands. extracting_data - data is being collected; building_dependencies - the tables are being built; ready_to_work - the last build finished and the output table can be queried; failed - the last build failed, see error. ready_to_work means built, not built completely: it is built from the extractions that delivered, and how complete that is shown by degraded and sources. New values can be added, so tolerate unknown ones.
degradedWhether the last build was incomplete. Only meaningful while state is ready_to_work: in every other state, a failed recipe and a running build included, it is false. true - the build left something out, either a whole data source or some extractions; see not_delivered_reasons and sources. false - everything in the build delivered. null - the completeness of the last build is unknown because that build ran before version 3.1.
not_delivered_reasonsThe distinct reasons behind an incomplete last build, sorted and deduplicated across all data sources. Empty when degraded is false or null.
errorWhy a build failed. On a recipe this is not null if and only if state is failed; on an entry of sources it is not null if and only if that entry's status is failed. Build error codes are a namespace of their own, separate from the codes of HTTP error responses - a few names appear in both. The catalogue is on the Errors page.
datatable_idThe recipe's output data table, or null while none is registered. Resolve the physical table name with GET /api/v3/data-tables/{data_table_id} instead of deriving it from a title.
last_build_atWhen the last build of this recipe was started, no matter what started it. null means the recipe has never been built. For recipes last built before version 3.1 this is an approximation: the time the recipe record last changed, which is often the end of that build rather than its start.
created_atWhen the recipe was created.
updated_atWhen the recipe itself last changed. It says nothing about builds - for those use last_build_at and state.
data_sourcesData source names the recipe is configured for, sorted. Empty on a recipe that has been created but not configured yet.
allowed_data_sourcesData source names the template allows, sorted. Everything in data_sources has to come from this list.
The connections and accounts the recipe reads. This is the full set the recipe holds; configuring a recipe always sends the full set, and nothing can be taken away from it.
sync_from_dateThe date the recipe collects data from. Always set in a response, even though it may be omitted from a configuration request.
What the last build did with each configured data source. One entry per name in data_sources.
Configure a recipe
Gives a recipe the data sources, connections and accounts it builds from. This step is mandatory: a recipe created through the API holds no configuration, and a build started before it answers 409 recipe_not_configured.
The body is the full set, not a delta: send every data source, connection and account the recipe should have, including the ones it already has. Take connection_id from GET /api/v3/datasources/{datasource_name}/connections and every account id from GET /api/v3.1/connections/{connection_id}/accounts/ of this same workspace, and use only accounts with is_active: true - anything else is rejected with 400 account_unavailable.
Configuration only grows. Data sources, connections and accounts can be added, and none of them can be taken away: a body that leaves one out is rejected with 409 and the matching *_removal_not_allowed code. An account cannot be removed from a recipe at all, by this operation or by any other, and deleting the recipe does not remove it either. If the data of an account has to be taken out of a result, contact support.
A successful call answers 200 with the recipe as it now stands, so the response can be compared with what was sent. It changes neither state nor error and starts no build; POST .../build/ does that.
path Parameters
recipe_idRecipe identifier, as returned when the recipe was created. A string of digits.
Configure a recipe › Request Body
data_sourcesEvery data source the recipe reads, by name. Cannot be empty, and every name has to be one of the recipe's allowed_data_sources. A data source already configured has to stay in the list.
Every connection the recipe reads, with its accounts. Cannot be empty. Each connection_data_source has to appear in data_sources, the pair of connection_id and connection_data_source has to be unique in the list, and every data source added by this call needs at least one connection here.
sync_from_dateThe first date the recipe collects data from. Omit it or send null to leave the date the recipe already has unchanged.
Configure a recipe › Responses
The recipe as it now stands, with the configuration this call saved
recipe_idRecipe identifier. A string, even though it is made of digits.
template_idThe template this recipe was created from.
template_titleCurrent name of that template.
titleRecipe name. When you create a recipe without a title, the template supplies the default.
workspace_idThe workspace this recipe lives in.
stateWhere the recipe stands. extracting_data - data is being collected; building_dependencies - the tables are being built; ready_to_work - the last build finished and the output table can be queried; failed - the last build failed, see error. ready_to_work means built, not built completely: it is built from the extractions that delivered, and how complete that is shown by degraded and sources. New values can be added, so tolerate unknown ones.
degradedWhether the last build was incomplete. Only meaningful while state is ready_to_work: in every other state, a failed recipe and a running build included, it is false. true - the build left something out, either a whole data source or some extractions; see not_delivered_reasons and sources. false - everything in the build delivered. null - the completeness of the last build is unknown because that build ran before version 3.1.
not_delivered_reasonsThe distinct reasons behind an incomplete last build, sorted and deduplicated across all data sources. Empty when degraded is false or null.
errorWhy a build failed. On a recipe this is not null if and only if state is failed; on an entry of sources it is not null if and only if that entry's status is failed. Build error codes are a namespace of their own, separate from the codes of HTTP error responses - a few names appear in both. The catalogue is on the Errors page.
datatable_idThe recipe's output data table, or null while none is registered. Resolve the physical table name with GET /api/v3/data-tables/{data_table_id} instead of deriving it from a title.
last_build_atWhen the last build of this recipe was started, no matter what started it. null means the recipe has never been built. For recipes last built before version 3.1 this is an approximation: the time the recipe record last changed, which is often the end of that build rather than its start.
created_atWhen the recipe was created.
updated_atWhen the recipe itself last changed. It says nothing about builds - for those use last_build_at and state.
data_sourcesData source names the recipe is configured for, sorted. Empty on a recipe that has been created but not configured yet.
allowed_data_sourcesData source names the template allows, sorted. Everything in data_sources has to come from this list.
The connections and accounts the recipe reads. This is the full set the recipe holds; configuring a recipe always sends the full set, and nothing can be taken away from it.
sync_from_dateThe date the recipe collects data from. Always set in a response, even though it may be omitted from a configuration request.
What the last build did with each configured data source. One entry per name in data_sources.
Delete a recipe
Deletes a recipe. This is a hard delete and it cannot be undone: the output tables the recipe produced are dropped, its configuration is removed, and the intermediate recipes it was built on go with it. The cascade follows the recipes this one reads from, not the ones that read it: a recipe built on top of this one stays in place and breaks.
The extractions created for the recipe are not deleted. They keep running on their schedule and the data they have already collected stays where it is, so a recipe deployed from the same template in the same workspace later sees the data of all those accounts, whatever set of accounts its own configuration names. Deleting a recipe is therefore not a way to narrow what a result contains; if the data of an account has to be taken out, contact support.
The template slot and the output table name are free again as soon as the 204 arrives, but the registration of the table on the platform side is removed asynchronously. Before deploying the same template here again, wait until the recipe is gone from GET /api/v3.1/automated-recipes/; a create sent straight after the 204 can be given the datatable_id of the old table, which is then dropped.
Answers 204 with an empty body. If the recipe itself could not be deleted, the answer is 500 delete_incomplete; an intermediate recipe that could not be deleted is logged on our side and does not change the answer.
path Parameters
recipe_idRecipe identifier, as returned when the recipe was created. A string of digits.
Delete a recipe › Responses
The recipe was deleted. The response has no body.
Start a recipe build
Starts a build of a configured recipe and answers 202 straight away with the recipe, whose last_build_at is the moment the build started. The request has no body.
The build runs in the background. Poll GET /api/v3.1/automated-recipes/{recipe_id}/ until state is ready_to_work or failed. There is no hard limit on how long a build takes, but in practice it finishes within 120 minutes; if there is still no outcome well after that, contact support with the x-im-trace-id of one of your requests.
ready_to_work means built, not built completely: a build runs on the extractions that delivered, so read degraded and sources as well. A recipe that has not been configured with PUT cannot be built and answers 409 recipe_not_configured. Starting a second build while one is running answers 409 build_already_running.
Use this operation again after a failed build, once the reason in error has been dealt with, and to pick up data that arrived late.
path Parameters
recipe_idRecipe identifier, as returned when the recipe was created. A string of digits.
Start a recipe build › Responses
The build was accepted and is running. The recipe comes back as it stands right now, with last_build_at set to the moment the build started.
recipe_idRecipe identifier. A string, even though it is made of digits.
template_idThe template this recipe was created from.
template_titleCurrent name of that template.
titleRecipe name. When you create a recipe without a title, the template supplies the default.
workspace_idThe workspace this recipe lives in.
stateWhere the recipe stands. extracting_data - data is being collected; building_dependencies - the tables are being built; ready_to_work - the last build finished and the output table can be queried; failed - the last build failed, see error. ready_to_work means built, not built completely: it is built from the extractions that delivered, and how complete that is shown by degraded and sources. New values can be added, so tolerate unknown ones.
degradedWhether the last build was incomplete. Only meaningful while state is ready_to_work: in every other state, a failed recipe and a running build included, it is false. true - the build left something out, either a whole data source or some extractions; see not_delivered_reasons and sources. false - everything in the build delivered. null - the completeness of the last build is unknown because that build ran before version 3.1.
not_delivered_reasonsThe distinct reasons behind an incomplete last build, sorted and deduplicated across all data sources. Empty when degraded is false or null.
errorWhy a build failed. On a recipe this is not null if and only if state is failed; on an entry of sources it is not null if and only if that entry's status is failed. Build error codes are a namespace of their own, separate from the codes of HTTP error responses - a few names appear in both. The catalogue is on the Errors page.
datatable_idThe recipe's output data table, or null while none is registered. Resolve the physical table name with GET /api/v3/data-tables/{data_table_id} instead of deriving it from a title.
last_build_atWhen the last build of this recipe was started, no matter what started it. null means the recipe has never been built. For recipes last built before version 3.1 this is an approximation: the time the recipe record last changed, which is often the end of that build rather than its start.
created_atWhen the recipe was created.
updated_atWhen the recipe itself last changed. It says nothing about builds - for those use last_build_at and state.
data_sourcesData source names the recipe is configured for, sorted. Empty on a recipe that has been created but not configured yet.
allowed_data_sourcesData source names the template allows, sorted. Everything in data_sources has to come from this list.
The connections and accounts the recipe reads. This is the full set the recipe holds; configuring a recipe always sends the full set, and nothing can be taken away from it.
sync_from_dateThe date the recipe collects data from. Always set in a response, even though it may be omitted from a configuration request.
What the last build did with each configured data source. One entry per name in data_sources.
Recreate a recipe from its template
Rebuilds the steps of an existing recipe from the current version of its template and starts a build of it, answering 202 straight away. The request has no body. Use it after the template has been changed in the interface, so a recipe deployed earlier picks the change up.
The configuration of the recipe is kept: data sources, connections, accounts and sync_from_date stay as they are, and so do the recipe identifier and the name of its output table. Nothing has to be sent again with PUT.
A build starts on its own, so no POST .../build/ is needed afterwards. Poll GET /api/v3.1/automated-recipes/{recipe_id}/ until state is ready_to_work or failed, as after any build.
path Parameters
recipe_idRecipe identifier, as returned when the recipe was created. A string of digits.
Recreate a recipe from its template › Responses
The recipe was recreated from its template and a build was accepted. The recipe comes back as it stands right now.
recipe_idRecipe identifier. A string, even though it is made of digits.
template_idThe template this recipe was created from.
template_titleCurrent name of that template.
titleRecipe name. When you create a recipe without a title, the template supplies the default.
workspace_idThe workspace this recipe lives in.
stateWhere the recipe stands. extracting_data - data is being collected; building_dependencies - the tables are being built; ready_to_work - the last build finished and the output table can be queried; failed - the last build failed, see error. ready_to_work means built, not built completely: it is built from the extractions that delivered, and how complete that is shown by degraded and sources. New values can be added, so tolerate unknown ones.
degradedWhether the last build was incomplete. Only meaningful while state is ready_to_work: in every other state, a failed recipe and a running build included, it is false. true - the build left something out, either a whole data source or some extractions; see not_delivered_reasons and sources. false - everything in the build delivered. null - the completeness of the last build is unknown because that build ran before version 3.1.
not_delivered_reasonsThe distinct reasons behind an incomplete last build, sorted and deduplicated across all data sources. Empty when degraded is false or null.
errorWhy a build failed. On a recipe this is not null if and only if state is failed; on an entry of sources it is not null if and only if that entry's status is failed. Build error codes are a namespace of their own, separate from the codes of HTTP error responses - a few names appear in both. The catalogue is on the Errors page.
datatable_idThe recipe's output data table, or null while none is registered. Resolve the physical table name with GET /api/v3/data-tables/{data_table_id} instead of deriving it from a title.
last_build_atWhen the last build of this recipe was started, no matter what started it. null means the recipe has never been built. For recipes last built before version 3.1 this is an approximation: the time the recipe record last changed, which is often the end of that build rather than its start.
created_atWhen the recipe was created.
updated_atWhen the recipe itself last changed. It says nothing about builds - for those use last_build_at and state.
data_sourcesData source names the recipe is configured for, sorted. Empty on a recipe that has been created but not configured yet.
allowed_data_sourcesData source names the template allows, sorted. Everything in data_sources has to come from this list.
The connections and accounts the recipe reads. This is the full set the recipe holds; configuring a recipe always sends the full set, and nothing can be taken away from it.
sync_from_dateThe date the recipe collects data from. Always set in a response, even though it may be omitted from a configuration request.
What the last build did with each configured data source. One entry per name in data_sources.
Check recipe extraction tables
Lists the extraction tables the recipe reads and says whether each of them exists in the destination of the workspace right now. The check runs against the destination on every call.
This answers "were the tables created", not "is the data ready". Readiness is the state of the recipe together with degraded and sources; a table can exist and still hold nothing useful. A data source whose extractions write into more than one table appears once per table, so key tables by table, not by data_source.
Use it when a build failed on missing data: if all_tables_exist is false, the extractions have not written anything yet and another build will fail the same way; if it is true, another build is worth starting.
path Parameters
recipe_idRecipe identifier, as returned when the recipe was created. A string of digits.
Check recipe extraction tables › Responses
The extraction tables of the recipe and whether they exist
all_tables_existtrue only when tables is not empty and every table in it exists right now. It does not mean the data is complete: for that read the recipe's state, degraded and sources.
One entry per extraction table the recipe reads. A data source with more than one table appears once per table, so key this list by table, not by data_source. Empty when the recipe reads no extraction tables yet.