Avatar SDK Unity Cloud plugin  3.0.1
Realistic avatar generation toolset for Unity3D
Scene #08: Full Body Parameters Sample

This sample has reacher functionality than the Full Body Getting Started sample and shows how to specify various parameters for avatar generation.

Detailed information about parameters can be found in our Web Api Documentation

Currently, the following groups with parameters are available:

  • Haircuts
  • Blendshapes
  • Outfits
  • Textures
  • Body Shape
  • Model Info
  • Avatar Modifications

Available Pipelines

There are three pipelines to generate full body avatars:

  • FitPerson (body_0.3/mobile) - generates models with an uniquie body shape. See Body Shape section to get more information about parameters.
  • MetaPerson Male (body_0.3/male) - generates male models with a fixed body shape. The body shape is the same for each avatar. Body Shape parameters are not available for this pipeline.
  • MetaPerson Female (body_0.3/female) - generates female models with a fixed body shape. The body shape is the same for each avatar. Body Shape parameters are not available for this pipeline.

Animations

This sample demonstrates how the full body model can be animated. There are two types of animations: facial animations via blendshapes and humanoid skeleton animations.
See Full Body Animations page to get detailed instructions on how to apply humanoid animation to the generated avatar.

Sample Of Usage

FullbodyAvatarComputationParameters is used to specify required parameters for full body avatar. It is passed to the IFullbodyAvatarProvider.InitializeFullbodyAvatarAsync method during avatar initialization.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, IFullbodyAvatarProvider fullbodyAvatarProvider)
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.haircuts.names.Add("generated");
computationParameters.outfits.names.Add("outfit_0");
computationParameters.modelInfo.age.Value = true;
computationParameters.modelInfo.skinColor.Value = true;
computationParameters.avatarModifications.removeGlasses.Value = true;
computationParameters.avatarModifications.removeSmile.Value = true;
computationParameters.avatarModifications.enhanceLighting.Value = true;
var initializeRequest = fullbodyAvatarProvider.InitializeFullbodyAvatarAsync(photoBytes, computationParameters, PipelineType.FIT_PERSON);
yield return initializeRequest;
}

Use IFullbodyAvatarProvider.GetAvailableComputationParametersAsync method to get all available parameters for the full body pipelne.

private IEnumerator GetAvailableParameters(IFullbodyAvatarProvider fullbodyAvatarProvider)
{
var parametersRequest = fullbodyAvatarProvider.GetAvailableComputationParametersAsync(PipelineType.FIT_PERSON);
yield return parametersRequest;
FullbodyAvatarComputationParameters availableParameters = parametersRequest.Result;
}

Parameters

LOD

We provide 8 LOD for the body mesh.
LOD 0: 35K triangles
LOD 1: 28K triangles
LOD 2: 25K triangles
LOD 3: 17.5K triangles
LOD 4: 14K triangles
LOD 5: 11.5K triangles
LOD 6: 10K triangles
LOD 7: 7K triangles

The LOD can be specified as follow:

private void ConfugureLOD()
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.lod = 7;
}

Haircuts

There is a set of artificial haircuts and the haircut generated from the original photo.
You can specify additional haircuts parameters:

  • Texture Size (default size is 2048x2048)
  • Haircut Color (isn't applied to the generated haircut)
  • Embed flag. This flag specify whether the haircuts meshes are embedded into the file with the avatar mesh or will be available as separate files. If the haircuts are not embedded they can be downloaded asynchronously.
private void ConfugureHaircutsParameters()
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.haircuts.names.Add("generated");
computationParameters.haircuts.names.Add("long_wavy");
computationParameters.haircuts.textureSize = new TextureSize(1024, 1024);
computationParameters.haircuts.color = new Color(0.0f, 1.0f, 0.0f);
computationParameters.haircuts.embed = false;
}

Blendshapes

There are two sets of blendshapes: mobile_51 and visemes_15.

private void ConfugureBlendshapesParameters()
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.blendshapes.names.Add("mobile_51");
computationParameters.blendshapes.names.Add("visemes_15");
}

Outfits

We provide 6 outfits:

Outfit 0 Outfit 2 Outfit 3
Business outfit, suits female avatars best. Business casual outfit, suits female avatars best. Casual outfit, suits female avatars best.
fb_outfit_0.jpg
fb_outfit_2.jpg
fb_outfit_3.jpg
Outfit 1 Outfit 4 Outfit 5
Casual outfit, suits male avatars best. Business casual outfit, suits male avatars best. Business outfit, suits male avatars best.
fb_outfit_1.jpg
fb_outfit_4.jpg
fb_outfit_5.jpg

Additional outfits parameters:

  • Texture Size (default value is 4096x4096). If you specify texture size, this value will be applied to all textures for all outfits. It isn't possible to set the texture size for each outfit independently.
  • Embed flag. This flag specifies whether the outfits meshes are embedded into the file with the avatar mesh or will be available as separate files. If the outfits are not embedded they can be downloaded asynchronously.
  • Additional textures: normal map, metallic map, roughness map, body_visibility_mask.

The body_visibility_mask texture is used to indicate which parts of the body are hidden under the particular outfit. It is useful to apply this mask when you are going to animate the full body model. Because in some poses the body mesh could be intersected with the outfit and the skin is rendered in front of the clothes. To get rid of such artifacts, the alpha channel of the body texture should be taken from this mask.

private void ConfugureOutfitsParameters()
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.outfits.names.Add("outfit_0");
computationParameters.outfits.names.Add("outfit_1");
computationParameters.outfits.embed = false;
computationParameters.outfits.textureSize = new TextureSize(2048, 2048);
//By default all additional textures are enabled, you can disable them as follow:
computationParameters.outfits.additionalTextures.Clear();
}

Lowpoly outfits
There is the set of the lowpoly outfits that correspond to the base outfits but has lower polygons count. See comparison table below.

Outfit name Base set Lowpoly Set
outfit_0 29815 vertices, 51616 faces 6186 vertices, 8880 faces
outfit_1 21426 vertices, 39312 faces 2872 vertices, 4270 faces
outfit_2 32570 vertices, 60102 faces 4252 vertices, 6844 faces
outfit_3 24655 vertices, 42832 faces 4983 vertices, 6626 faces
outfit_4 17830 vertices, 30383 faces 4518 vertices, 6270 faces
outfit_5 27715 vertices, 48362 faces 5875 vertices, 9017 faces

Textures

There are additional textures for the body mesh that are enabled by default: normal_map, roughness_map, metallic_map.
These textures are the same size that can be configured (default value is 4096x2048).

private void ConfugureTexturesParameters()
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.textureSize = new TextureSize(2048, 2048);
//If you don't need additional textures use code below
computationParameters.additionalTextures.names.Clear();
}

Body Shape

This group of parameters is used to configure body shape.

  • Gender - expected model gender. Available values are: female, male and non_binary.
  • Height - expected model height in meters, greater than zero.
  • Weight - expected model weight in kilograms, greater than zero.
  • Chest - expected model chest girth in centimeters, greater than zero.
  • Waist - expected model waist girth in centimeters, greater than zero.
  • Hips - expected model hips girth in centimeters, greater than zero.

Avatar SDK algorithms predict full body shape by analyzing an input photo and producing an average body for detected parameters. Parameters that were not specified will be predicted from an input photo. However, it is recommended to specify as many parameters as possible to get a specific body shape.

Please note: the output model may not necessarily have exactly the same measurements as provided by the avatar request. The output measurements are expected to be close to the input body shape parameters that are within the intervals specified by the table below. Values outside of these intervals may result in significantly different output measurements. We are working to expand these intervals, specifically the one corresponding to the height parameter.

Parameter Name Female Male Non Binary
height [150; 174] [160; 187] [150; 187]
weight [54; 86] [70; 112] [54; 112]
chest [90; 115] [96; 125] [90; 125]
waist [73; 98] [81; 118] [73; 118]
hips [90; 116] [93; 119] [90; 119]

Model Info

There are following parameters from the Model Info group available for the full body avatars:

  • Hair Color: Compute average haircut color from the submitted photo.
  • Skin Color: Compute average skin color from the submitted photo.
  • Gender: Predict a person's gender from the submitted photo (male or female) and confidence.
  • Age: Classify to which age group the person from the photo belongs to. There are two groups: Child and Adult.
  • Facial Landmarks: Compute facial landmarks.
  • Eye Sclera Color: Compute average eye sclera color from the submitted photo.
  • Eye Iris Color: Compute average eye iris color from the submitted photo.
  • Race: Classify to which race the person from the photo belongs to. Possible values are: black, asian and white. The result of prediction is represented as a set of confidence values for possible values.

More information about Model Info parameters can be found in our Web Api Documentation

Avatar Modifications

There are following parameters from the Avatar Modifications group available for the full body avatars:

  • Remove Smile: Detect smile on input photo and remove it if detected.
  • Remove Glasses: Remove glasses on the submitted photo if detected.
  • Remove Stubble: Remove stubble on the submitted photo if detected.
  • Enhance Lighting: Attempt to make lighting more uniform.
  • Parametric eyes texture: Replace eye texture from submitted photo to generated one with sclera and iris colors match the photo version.
  • Eye Iris Color: Allow to set the iris color. Works only when parametric_eyes_texture is set.
  • Eye Sclera Color: Allow to set the eye sclera color. Works only when parametric_eyes_texture is set.
  • Teeth Color: Recolor teeth directly on a model texture.
  • Generated haircut texture size: Size of generated haircut's texture.
  • Generated haircut faces count: Number of faces in the generated haircut.

More information about Avatar Modifications parameters can be found in Parameters Sample page or in Web Api Documentation

Template parameter

You can generate only the head of the avatar without a body. Set the template parameter to the ExportTemplate.HEAD for it.

private void ConfugureTemplate()
{
FullbodyAvatarComputationParameters computationParameters = new FullbodyAvatarComputationParameters();
computationParameters.template = ExportTemplate.HEAD;
}

Implementation

Implementation details of this sample can be found in the FullbodyParametersSample.cs script.

Location: Assets/itseez3d/avatar_sdk/samples_cloud/08_fullbody_parameters_sample_cloud/scenes/08_fullbody_parameters_sample_cloud.unity.