Workflows
To use the Unstructured Workflow Endpoint to manage workflows, do the following:
- To get a list of available workflows, use the
UnstructuredClient
object’sworkflows.list_workflows
function (for the Python SDK) or theGET
method to call the/workflows
endpoint (forcurl
or Postman). Learn more. - To get information about a workflow, use the
UnstructuredClient
object’sworkflows.get_workflow
function (for the Python SDK) or theGET
method to call the/workflows/<workflow-id>
endpoint (forcurl
or Postman)use theGET
method to call the/workflows/<workflow-id>
endpoint. Learn more. - To create a workflow, use the
UnstructuredClient
object’sworkflows.create_workflow
function (for the Python SDK) or thePOST
method to call the/workflows
endpoint (forcurl
or Postman). Learn more. - To run a workflow manually, use the
UnstructuredClient
object’sworkflows.run_workflow
function (for the Python SDK) or thePOST
method to call the/workflows/<workflow-id>/run
endpoint (forcurl
or Postman). Learn more. - To update a workflow, use the
UnstructuredClient
object’sworkflows.update_workflow
function (for the Python SDK) or thePUT
method to call the/workflows/<workflow-id>
endpoint (forcurl
or Postman). Learn more. - To delete a workflow, use the
UnstructuredClient
object’sworkflows.delete_workflow
function (for the Python SDK) or theDELETE
method to call the/workflows/<workflow-id>
endpoint (forcurl
or Postman). Learn more.
The following examples assume that you have already met the requirements and understand the basics of working with the Unstructured Workflow Endpoint.
Create a workflow
To create a workflow, use the UnstructuredClient
object’s workflows.create_workflow
function (for the Python SDK) or
the POST
method to call the /workflows
endpoint (for curl
or Postman).
In the CreateWorkflow
object (for the Python SDK) or
the request body (for curl
or Postman),
specify the settings for the workflow, as follows:
Replace the preceding placeholders as follows:
-
<name>
(required) - A unique name for this workflow. -
<source-connector-id>
(required) - The ID of the target source connector. To get the ID, use theUnstructuredClient
object’ssources.list_sources
function (for the Python SDK) or theGET
method to call the/sources
endpoint (forcurl
or Postman). Learn more. -
<destination-connector-id>
(required) - The ID of the target destination connector. To get the ID, use theUnstructuredClient
object’sdestinations.list_destinations
function (for the Python SDK) or theGET
method to call the/destinations
endpoint (forcurl
or Postman). Learn more. -
<TYPE>
(for the Python SDK) or<type>
(forcurl
or Postman) (required) - The workflow type. Available values includeCUSTOM
(for the Python SDK) andcustom
(forcurl
or Postman).If
<TYPE>
is set toCUSTOM
(for the Python SDK), or if<type>
is set tocustom
(forcurl
or Postman), you must add aworkflow_nodes
array. For instructions, see Custom workflow DAG nodes.The previously-available workflow optimization types
ADVANCED
,BASIC
, andPLATINUM
(for the Python SDK) andadvanced
,basic
, andplatinum
(forcurl
or Postman) are non-operational and planned to be fully removed in a future release.The ability to create an automatic workflow type is currently not available but is planned to be added in a future release.
-
<schedule-timeframe>
- The repeating automatic run schedule, specified as a predefined phrase. The available predefined phrases are:every 15 minutes
(forcurl
or Postman): Every 15 minutes (cron expression:*/15 * * * *
).every hour
: At the first minute of every hour (cron expression:0 * * * *
).every 2 hours
: At the first minute of every second hour (cron expression:0 */2 * * *
).every 4 hours
: At the first minute of every fourth hour (cron expression:0 */4 * * *
).every 6 hours
: At the first minute of every sixth hour (cron expression:0 */6 * * *
).every 8 hours
: At the first minute of every eighth hour (cron expression:0 */8 * * *
).every 10 hours
: At the first minute of every tenth hour (cron expression:0 */10 * * *
).every 12 hours
: At the first minute of every twelfth hour (cron expression:0 */12 * * *
).daily
: At the first minute of every day (cron expression:0 0 * * *
).weekly
: At the first minute of every Sunday (cron expression:0 0 * * 0
).monthly
: At the first minute of the first day of every month (cron expression:0 0 1 * *
).
If
schedule
is not specified, the workflow does not automatically run on a repeating schedule.Workflows with a local source cannot be set to run on a repeating schedule.
Update a workflow
To update information about a workflow, use the UnstructuredClient
object’s workflows.update_workflow
function (for the Python SDK) or
the PUT
method to call the /workflows/<workflow-id>
endpoint (for curl
or Postman), replacing
<workflow-id>
with the workflow’s unique ID. To get this ID, see List workflows.
In the request body, specify the settings for the workflow. For the specific settings to include, see Create a workflow.
Custom workflow DAG nodes
If WorkflowType
is set to CUSTOM
(for the Python SDK), or if workflow_type
is set to custom
(for curl
or Postman), you must also specify the settings for the workflow’s
directed acyclic graph (DAG) nodes. These nodes’ settings are specified in the workflow_nodes
array.
-
A Source node is automatically created when you specify the
source_id
value outside of the
workflow_nodes
array. -
A Destination node is automatically created when you specify the
destination_id
value outside of theworkflow_nodes
array. -
You can specify Partitioner, Chunker, Enrichment, and Embedder nodes.
Image summary descriptions, table summary descriptions, and table-to-HTML output is generated only when the Partitioner node in a workflow is set to use the High Res partitioning strategy and the workflow also contains an image description, table description, or table-to-HTML enrichment node.
Setting the Partitioner node to use Auto, VLM, or Fast in a workflow that also contains an image description, table description, or table-to-HTML enrichment node will not generate any image summary descriptions, table summary descriptions, or table-to-HTML output, and it could also cause the workflow to stop running or produce unexpected results.
-
The order of the nodes in the
workflow_nodes
array will be the same order that these nodes appear in the DAG, with the first node in the array added directly after the Source node. The Destination node follows the last node in the array. -
Be sure to specify nodes in the allowed order. The following DAG placements are all allowed:
Partitioner node
A Partitioner node has a type
of partition
.
Learn about the available partitioning strategies.
Auto strategy
Fields for settings
include:
-
strategy
: Required. The partitioning strategy to use. This field must be set toauto
. -
provider
: Optional. If the Auto partitioning strategy needs to use the VLM partitioning strategy, then use the specified VLM provider. Allowed values includeauto
,openai
,anthropic
, andbedrock
. The default value isanthropic
. -
provider_api_key
: Optional. If specified, use a non-default API key for calls to the specified VLM provider as needed. The default is none, which means to rely on using Unstructured’s internal default API key for the VLM provider. -
model
: Optional. If the Auto partitioning strategy needs to use the VLM partitioning strategy, then use the specified VLM. The default value isclaude-3-5-sonnet-20241022
.-
For
openai
, available values formodel
aregpt-4o
andgpt-4o-mini
. -
For
anthropic
, available values formodel
areclaude-3-5-sonnet-20241022
andclaude-3-7-sonnet-20250219
. -
For
bedrock
, available values formodel
are:us.amazon.nova-lite-v1:0
us.amazon.nova-pro-v1:0
us.anthropic.claude-3-opus-20240229-v1:0
us.anthropic.claude-3-haiku-20240307-v1:0
us.anthropic.claude-3-sonnet-20240229-v1:0
us.anthropic.claude-3-5-sonnet-20241022-v2:0
us.meta.llama3-2-11b-instruct-v1:0
us.meta.llama3-2-90b-instruct-v1:0
-
-
output_format
: Output. The format of the response. Allowed values includetext/html
andapplication/json
. The default istext/html
. -
prompt.text
: Optional. If the Auto partitioning strategy needs to use the VLM partitioning strategy, then use the specified prompt when calling the specified VLM. The default value is none, which means to rely on using Unstructured’s internal default prompt when calling the VLM. -
format_html
: Optional. If the Auto partitioning strategy needs to use the VLM partitioning strategy, true (the default) to apply Beautiful Soup’sprettify
method to the HTML that is generated by the VLM partitioner, which for example adds indentation for better readability. -
unique_element_ids
: Optional. True (the default) to assign UUIDs to element IDs, which guarantees their uniqueness. This is useful for example when using them as primary keys in a database. False to assign a SHA-256 of the element’s text as its element ID. -
is_dynamic
: Optional. True (the default) to enable dynamic routing of pages to Fast, High Res, or VLM as needed for better overall performance and cost savings. -
allow_fast
: Optional. True (the default) to allow routing of pages to Fast as needed for better overall performance and cost savings.
VLM strategy
Fields for settings
include:
-
provider
: Optional. Use the specified VLM provider. Allowed values includeauto
,openai
,anthropic
, andbedrock
. The default value isanthropic
. -
provider_api_key
: Optional. If specified, use a non-default API key for calls to the specified VLM provider as needed. The default is none, which means to rely on using Unstructured’s internal default API key for the VLM provider. -
model
: Optional. If the Auto partitioning strategy needs to use the VLM partitioning strategy, then use the specified VLM. The default value isclaude-3-5-sonnet-20241022
.-
For
openai
, available values formodel
aregpt-4o
andgpt-4o-mini
. -
For
anthropic
, available values formodel
areclaude-3-5-sonnet-20241022
andclaude-3-7-sonnet-20250219
. -
For
bedrock
, available values formodel
are:us.amazon.nova-lite-v1:0
us.amazon.nova-pro-v1:0
us.anthropic.claude-3-opus-20240229-v1:0
us.anthropic.claude-3-haiku-20240307-v1:0
us.anthropic.claude-3-sonnet-20240229-v1:0
us.anthropic.claude-3-5-sonnet-20241022-v2:0
us.meta.llama3-2-11b-instruct-v1:0
us.meta.llama3-2-90b-instruct-v1:0
-
-
output_format
: Output. The format of the response. Allowed values includetext/html
andapplication/json
. The default istext/html
. -
prompt.text
: Optional. Use the specified prompt when calling the specified VLM. The default value is none, which means to rely on using Unstructured’s internal default prompt when calling the VLM. -
format_html
: Optional. True (the default) to apply Beautiful Soup’sprettify
method to the HTML that is generated by the VLM partitioner, which for example adds indentation for better readability. -
unique_element_ids
: Optional. True (the default) to assign UUIDs to element IDs, which guarantees their uniqueness. This is useful for example when using them as primary keys in a database. False to assign a SHA-256 of the element’s text as its element ID. -
is_dynamic
: Optional. This setting has no effect for the VLM strategy. The default is false. -
allow_fast
: Optional. This setting has no effect for the VLM strategy. The default is true.
High Res strategy
-
strategy
: Required. The partitioning strategy to use. This field must be set tohi_res
. -
include_page_breaks
: Optional. True to include page breaks in the output if supported by the file type. The default is false. -
pdf_infer_table_structure
: Optional. True for anyTable
elements extracted from a PDF to include an additional metadata field,text_as_html
, where the value (string) is a just a transformation of the data into an HTML table. The default is false. -
exclude_elements
: Optional. A list of any Unstructured element types to exclude from the output. The default is none. Available values include:FigureCaption
NarrativeText
ListItem
Title
Address
Table
PageBreak
Header
Footer
UncategorizedText
Image
Formula
EmailAddress
-
xml_keep_tags
: Optional. True to retain any XML tags in the output. False (the default) to just extract the text from any XML tags instead. -
encoding
: Optional. The encoding method used to decode the text input. The default isutf-8
. -
ocr_languages
: Optional. A list of languages present in the input, for use in partitioning, OCR, or both. Multiple languages indicate that the text could be in any of the specified languages. The default is[ 'eng' ]
. See the language codes list. -
extract_image_block_types
: Optional. A list of the Unstructured element types for use in extracting image blocks as Base64 encoded data stored inmetadata
fields. Available values includeImage
andTable
. The default is[ 'Image', 'Table' ]
. -
infer_table_structure
: Optional. True to have any table elements extracted from a PDF to include an additionalmetadata
field namedtext_as_html
, containing an HTML<table>
transformation. The default is false.
Fast strategy
Fields for settings
include:
-
strategy
: Required. The partitioning strategy to use. This field must be set tofast
. -
`include_page_breaks: Optional. True to include page breaks in the output if supported by the file type. The default is false.
-
pdf_infer_table_structure
: Optional. Although this field is listed, it applies only to thehi_res
strategy and will not work if set to true. The default is false. -
exclude_elements
: Optional. A list of any Unstructured element types to exclude from the output. The default is none. Available values include:FigureCaption
NarrativeText
ListItem
Title
Address
Table
PageBreak
Header
Footer
UncategorizedText
Image
Formula
EmailAddress
-
xml_keep_tags
: Optional. True to retain any XML tags in the output. False (the default) to just extract the text from any XML tags instead. -
encoding
: Optional. The encoding method used to decode the text input. The default isutf-8
. -
ocr_languages
: Optional. A list of languages present in the input, for use in partitioning, OCR, or both. Multiple languages indicate that the text could be in any of the specified languages. The default is[ 'eng' ]
. See the language codes list. -
extract_image_block_types
: Optional. A list of the Unstructured element types for use in extracting image blocks as Base64 encoded data stored inmetadata
fields. Available values includeImage
andTable
. The default is[ 'Image', 'Table' ]
. -
infer_table_structure
: Optional. True to have any table elements extracted from a PDF to include an additionalmetadata
field namedtext_as_html
, containing an HTML<table>
transformation. The default is false.
Chunker node
A Chunker node has a type
of chunk
.
Learn about the available chunking strategies.
Chunk by Character strategy
Fields for settings
include:
unstructured_api_url
: Optional. If specified, use a non-default API URL for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API URL for the chunker.unstructured_api_key
: Optional. If specified, use a non-default API key for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API key for the chunker.include_orig_elements
: Optional. True to have the elements that are used to form a chunk appear in.metadata.orig_elements
for that chunk. The default is false.new_after_n_chars
: Optional. Closes new sections after reaching a length of this many characters. This is an approximate limit. The default is none.max_characters
: Optional. The absolute maximum number of characters in a chunk. The default is none.overlap
: Optional. Applies a prefix of this many trailing characters from the prior text-split chunk to second and later chunks formed from oversized elements by text-splitting. The default is none.overlap_all
: Optional. True to apply overlap to “normal” chunks formed by combining whole elements. Use with caution as this can introduce noise into otherwise clean semantic units. The default is false.contextual_chunking_strategy
: Optional. If specified, prepends chunk-specific explanatory context to each chunk. Allowed values includev1
. The default is none.
Chunk by Title strategy
Fields for settings
include:
unstructured_api_url
: Optional. If specified, use a non-default API URL for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API URL for the chunker.unstructured_api_key
: Optional. If specified, use a non-default API key for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API key for the chunker.-multipage_sections
: Optional. … The default is false.combine_text_under_n_chars
: Optional. Combines elements from a section into a chunk until a section reaches a length of this many characters. The default is none.include_orig_elements
: Optional. True to have the elements that are used to form a chunk appear in.metadata.orig_elements
for that chunk. The default is false.new_after_n_chars
: Optional. Closes new sections after reaching a length of this many characters. This is an approximate limit. The default is none.max_characters
: Optional. The absolute maximum number of characters in a chunk. The default is none.overlap
: Optional. Applies a prefix of this many trailing characters from the prior text-split chunk to second and later chunks formed from oversized elements by text-splitting. The default is none.overlap_all
: Optional. True to apply overlap to “normal” chunks formed by combining whole elements. Use with caution as this can introduce noise into otherwise clean semantic units. The default is false.contextual_chunking_strategy
: Optional. If specified, prepends chunk-specific explanatory context to each chunk. Allowed values includev1
. The default is none.
Chunk by Page strategy
Fields for settings
include:
unstructured_api_url
: Optional. If specified, use a non-default API URL for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API URL for the chunker.unstructured_api_key
: Optional. If specified, use a non-default API key for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API key for the chunker.-include_orig_elements
: Optional. … The default is false.include_orig_elements
: Optional. True to have the elements that are used to form a chunk appear in.metadata.orig_elements
for that chunk. The default is false.new_after_n_chars
: Optional. Closes new sections after reaching a length of this many characters. This is an approximate limit. The default is none.max_characters
: Optional. The absolute maximum number of characters in a chunk. The default is none.overlap
: Optional. Applies a prefix of this many trailing characters from the prior text-split chunk to second and later chunks formed from oversized elements by text-splitting. The default is none.overlap_all
: Optional. True to apply overlap to “normal” chunks formed by combining whole elements. Use with caution as this can introduce noise into otherwise clean semantic units. The default is false.contextual_chunking_strategy
: Optional. If specified, prepends chunk-specific explanatory context to each chunk. Allowed values includev1
. The default is none.
Chunk by Similarity strategy
Fields for settings
include:
unstructured_api_url
: Optional. If specified, use a non-default API URL for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API URL for the chunker.unstructured_api_key
: Optional. If specified, use a non-default API key for calls to the specified chunker as needed. The default is none, which means to rely on using Unstructured’s internal default API key for the chunker.include_orig_elements
: Optional. True to have the elements that are used to form a chunk appear in.metadata.orig_elements
for that chunk. The default is false.new_after_n_chars
: Optional. Closes new sections after reaching a length of this many characters. This is an approximate limit. The default is none.max_characters
: Optional. The absolute maximum number of characters in a chunk. The default is none.overlap
: Optional. Applies a prefix of this many trailing characters from the prior text-split chunk to second and later chunks formed from oversized elements by text-splitting. The default is none.overlap_all
: Optional. True to apply overlap to “normal” chunks formed by combining whole elements. Use with caution as this can introduce noise into otherwise clean semantic units. The default is false.contextual_chunking_strategy
: Optional. If specified, prepends chunk-specific explanatory context to each chunk. Allowed values includev1
. The default is none.similarity_threshold
: Optional. The minimum similarity that text in consecutive elements must have to be included in the same chunk. This must be a value between0.0
and1.0
, exclusive (0.01
to0.99
). The default is none.
Enrichment node
An Enrichment node has a type
of prompter
.
Learn about the available enrichments.
Image summary descriptions, table summary descriptions, and table-to-HTML output is generated only when the Partitioner node in a workflow is set to use the High Res partitioning strategy and the workflow also contains an image description, table description, or table-to-HTML enrichment node.
Setting the Partitioner node to use Auto, VLM, or Fast in a workflow that also contains an image description, table description, or table-to-HTML enrichment node will not generate any image summary descriptions, table summary descriptions, or table-to-HTML output, and it could also cause the workflow to stop running or produce unexpected results.
Image Description task
Image summary descriptions are generated only when the Partitioner node in a workflow is set to use the High Res partitioning strategy and the workflow also contains an image description enrichment node.
Setting the Partitioner node to use Auto, VLM, or Fast in a workflow that also contains an image description enrichment node will not produce any image summary descriptions, and it could also cause the workflow to stop running or produce unexpected results.
Allowed values for <subtype>
include:
openai_image_description
anthropic_image_description
bedrock_image_description
Table Description task
Table summary descriptions are generated only when the Partitioner node in a workflow is set to use the High Res partitioning strategy and the workflow also contains a table description enrichment node.
Setting the Partitioner node to use Auto, VLM, or Fast in a workflow that also contains a table description enrichment node will not produce any table summary descriptions, and it could also cause the workflow to stop running or produce unexpected results.
Allowed values for <subtype>
include:
openai_table_description
anthropic_table_description
bedrock_table_description
Table to HTML task
Table-to-HTML generation happens only when the Partitioner node in a workflow is set to use the High Res partitioning strategy and the workflow also contains a table-to-HTML enrichment node.
Setting the Partitioner node to use Auto, VLM, or Fast in a workflow that also contains a table-to-HTML enrichment node will not generate any table-to-HTML output, and it could also cause the workflow to stop running or produce unexpected results.
Named Entity Recognition (NER) task
Fields for settings include:
prompt_interface_overrides.prompt.user
: Optional. Any alternative prompt to use with the underlying NER model. The default is none, which means to rely on using Unstructured’s internal default prompt when calling the NER model.
Allowed values for <subtype>
include:
openai_ner
anthropic_ner
Embedder node
An Embedder node has a type
of embed
.
Learn about the available embedding providers and models.
Allowed values for subtype
and model_name
include:
-
"subtype": "azure_openai"
"model_name": "text-embedding-3-small"
"model_name": "text-embedding-3-large"
"model_name": "text-embedding-ada-002"
-
"subtype": "bedrock"
"model_name": "amazon.titan-embed-text-v2:0"
"model_name": "amazon.titan-embed-text-v1"
"model_name": "amazon.titan-embed-image-v1"
"model_name": "cohere.embed-english-v3"
"model_name": "cohere.embed-multilingual-v3"
-
"subtype": "togetherai"
"model_name": "togethercomputer/m2-bert-80M-2k-retrieval"
"model_name": "togethercomputer/m2-bert-80M-8k-retrieval"
"model_name": "togethercomputer/m2-bert-80M-32k-retrieval"
-
"subtype": "voyageai"
"model_name": "voyage-3"
"model_name": "voyage-3-large"
"model_name": "voyage-3-lite"
"model_name": "voyage-code-3"
"model_name": "voyage-finance-2"
"model_name": "voyage-law-2"
"model_name": "voyage-code-2"
"model_name": "voyage-multimodal-3"