Changing Digital Human Backgrounds

This document describes how to use the API to replace the background of a Digital Human with a static image. This feature allows for greater customization and flexibility in how Digital Humans are presented.

Overview

The process of changing the background involves several steps, including identifying head visual ID with repleceable background, uploading an image, triggering the background replacement and finally creating and configuring a head visual.

Process flow:

The following endpoints are used in the background replacement process:

1. Identify a head visual ID with replaceable background

We've created head visuals with associated assets that allow easy background replacement. Please identify the head visual ID you would like to replace background to using the folowing table:

If you are interested in creating a completely new head visual with replaceable background, please contact us.

2. Upload Replacement Image

• Endpoint: /image/upload
• Method: POST
• Description: Uploads the static image that will be used to replace the background.
• Request Body: (multipart/form-data)

curl -X 'POST' \
  'https://platform-api.unith.ai/image/upload' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer yourBearerToken \
  -H 'Content-Type: multipart/form-data' \
  -F 'file=@dolphinWallpaper.jpg;type=image/jpeg'

• Response:
  • token: A temporary token for the uploaded image.
• Error Handling:
  • Invalid file type (.png, .jpg, .jpeg are allowed).
  • File size exceeds the maximum allowed limit.

3. Replace Background

• Endpoint: /head_visual/replace-background
• Method: POST
• Description: Initiates the background replacement process.
• Request Body:

curl -X 'POST' \
  'https://platform-api.unith.ai/head_visual/replace-background' \
  -H 'accept: application/json' \
  -H 'x-head-image-token-id: imageToken \
  -H 'Authorization: Bearer yourBearerToken \
  -H 'Content-Type: application/json' \
  -d '{
  "id": "yourHeadVisualId"
}'

• Parameters:
  • x-head-image-token-id (header)(string, required): The token of the uploaded replacement image.
  • id: The ID of one of the head visuals from the table above.
• Response:
  • A JSON object containing the URL of the new idle video with the replaced background. For example:
• Error Handling: Handle errors related to the background replacement, including:
  • Invalid head visual ID.
  • Invalid image token.

4. Upload new video with replaced background

• Endpoint: /video/upload
• Method: POST
• Description: Upload the new video with replaced background.
• Request Body: (multipart/form-data)

curl -X 'POST' \
  'https://platform-api.unith.ai/video/upload' \
  -H 'accept: application/json' \
  -H 'Authorization: yourBearerToken \
  -H 'Content-Type: multipart/form-data' \
  -F 'file=@yourVideo.mp4;type=video/mp4'

• Response:
  • token: A temporary token for the uploaded video. This token is used in subsequent steps.
• Error Handling:
  • Invalid file type (.mp4 allowed).
  • File size exceeds the maximum allowed limit.

5. Create Head Visual with new replaced background.

• Endpoint: /head_visual/create
• Method: POST
• Description: Creates a new head visual using the video with replaced background.
• Request Body:

curl -X 'POST' \
  'https://platform-api.unith.ai/head_visual/create' \
  -H 'accept: application/json' \
  -H 'x-head-video-token-id: yourVideoToken' \
  -H 'Authorization: Bearer yourBearerToken' \
  -H 'Content-Type: application/json' \
  -d '{
  "update": false,
  "detector_version": "v2",
  "detector_threshold": -0.2,
  "mode": "default",
  "cut_timestamp": 0.1,
  "debug": false
}'

• Parameters:
  • Include all the required parameters for the /head_visual/create endpoint.
  • detector_version (string): You may or may not need to include it here, depending on your logic.
• Response:
  • A JSON object containing the details of the newly created head visual, including its unique ID.
• Error Handling:
  • Invalid video token.
  • Ensure cut_timestamp is provided when the processor mode is two_loops.

6. Save Head Visual.

• Endpoint: /head_visual/save
• Method: POST
• Description: Saves the newly created head visual.
• Request Body:

curl -X 'POST' \
  'https://platform-api.unith.ai/head_visual/save' \
  -H 'accept: application/json' \
  -H 'Authorization: yourBearerToken' \
  -H 'Content-Type: application/json' \
  -d '{
  "id": "yourHeadVisualID",
  "name": "headVisualName",
  "gender": "FEMALE",
  "type": "TALK"
}'

• Parameters:
  • Include all the required parameters for the /head_visual/save endpoint.
• Response:
  • A JSON object indicating the head visual has been saved.
• Error Handling:
  • Ensure the task exist for a given head_visual_id.
  • Check the task status is IN_PROGRESS.

7. Assign new head visual ID to your organization

• Endpoint: /head_visual/assign-organisation
• Method: PUT
• Description: Assigns the newly created head visual toy our organization
• Request Body:

curl -X 'PUT' \
  'https://platform-api.unith.ai/head_visual/assign-organisation' \
  -H 'accept: */*' \
  -H 'Authorization: Bearer yourBearerToken \
  -H 'Content-Type: application/json' \
  -d '{
  "id": "yourHeadVisualId",
  "orgIds": [
    "yourOrgId"
  ]
}'

• Parameters:
  • Include all the required parameters for the /head_visual/assign-organisation endpoint.
• Response:
  • A JSON object indicating the head visual has been assigned.
• Error Handling:
  • Ensure the task exist for a given head_visual_id and org_id.

Other Considerations

• Asynchronous Processing: Background replacement involves synchronous processing. Head visual creation, preprocessing of the video etc.