Scene #04: Parameters Sample

This scene demonstrates API that allows working with computation parameters and flags. These parameters and flags configure which auxiliary features and resources should be generated for the avatar.

The SDK will tell you which resources and parameters are available for your account (this is the full list that you see in the UI). Then before the avatar is generated you can specify which resources you need and which parameters you would like to apply.

Detailed information about parameters can be found in our Web Api Documentation

Accounts with the paid subscription plan will have more resources available and more extended parameters for their avatars.

Currently, the following groups with parameters are available:

• Haircuts
• Blendshapes
• Model Info
• Avatar Modifications
• Shape Modifications
• Additional Textures

Code Usage

ComputationParameters is used to specify required parameters. It is passed to the IAvatarProvider.InitializeAvatarAsync method during avatar initialization.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, PipelineType pipeline)
{
    ComputationParameters computationParameters = ComputationParameters.Empty;

    computationParameters.modelInfo = new ModelInfoGroup();
    computationParameters.modelInfo.age.Value = true;
    computationParameters.modelInfo.skinColor.Value = true;

    computationParameters.avatarModifications = new AvatarModificationsGroup();
    computationParameters.avatarModifications.removeGlasses.Value = true;
    computationParameters.avatarModifications.removeSmile.Value = true;
    computationParameters.avatarModifications.enhanceLighting.Value = true;

    var initializeRequest = avatarProvider.InitializeAvatarAsync(photoBytes, string.Empty, null, pipeline, computationParameters);
    yield return initializeRequest;
}

Use IAvatarProvider.GetParametersAsync method to get all available parameters for the given pipelne.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, PipelineType pipeline)
{
    var parametersRequest = avatarProvider.GetParametersAsync(ComputationParametersSubset.ALL, pipeline);
    yield return parametersRequest;

    ComputationParameters availableParameters = parametersRequest.Result;

    ComputationParameters specifiedParameters = ComputationParameters.Empty;

    specifiedParameters.modelInfo = new ModelInfoGroup();
    if (availableParameters.modelInfo.predictHaircut.IsAvailable)
        specifiedParameters.modelInfo.predictHaircut.Value = true;
    if (availableParameters.modelInfo.hairColor.IsAvailable)
        specifiedParameters.modelInfo.hairColor.Value = true;

    specifiedParameters.avatarModifications = new AvatarModificationsGroup();
    if (availableParameters.avatarModifications.removeGlasses.IsAvailable)
        specifiedParameters.avatarModifications.removeGlasses.Value = true;
    if (availableParameters.avatarModifications.removeSmile.IsAvailable)
        specifiedParameters.avatarModifications.removeSmile.Value = true;
    if (availableParameters.avatarModifications.enhanceLighting.IsAvailable)
        specifiedParameters.avatarModifications.enhanceLighting.Value = true;

    specifiedParameters.blendshapes = availableParameters.blendshapes;
    specifiedParameters.haircuts = availableParameters.haircuts;
    specifiedParameters.additionalTextures = availableParameters.additionalTextures;

    var initializeRequest = avatarProvider.InitializeAvatarAsync(photoBytes, string.Empty, null, pipeline, specifiedParameters);
    yield return initializeRequest;
}

Parameters

Haircuts

Unity plugin includes the following sets of artificial haircuts:

• Base set: 3 male and 3 female haircuts
• Facegen set: 39 haircuts in total, some of them are unisex
• Plus set: set of haircuts available for HEAD 2.0 pipeline
• Generated: haircut generated from the original image (available for HEAD 2.0 pipeline)

It is possible to add custom haircuts to your account. Please contact us at suppo.nosp@m.rt@a.nosp@m.vatar.nosp@m.sdk..nosp@m.com to implement this, we will provide the implementation details on a custom basis. Once the custom haircuts are added, the SDK will return more haircuts in the available haircuts list.

Blendshapes

Avatar SDK supports avatars animation using blendshapes. There are two different blendshapes sets available in the Avatar SDK Unity plugin which should be used depending on your use-case: general animation and lip-sync. E.g. if you want to use lipsync functionality (such as Oculus Lipsync SDK) with your avatar you may want to generate additional visemes blendshapes, but if you don't use lipsync you should tell the SDK to skip these blendshapes to save calculation and download time.

• legacy_45: Basic set of 45 facial blendshapes.
• visemes_17: Set of 15 visemes (aa, ch, ee, ih, oh, ou, th, dd, ff, kk, nn, pp, rr, sil, ss) and two additional blendshapes: frown and smile. This set is compatible with Oculus Lipsync that required these 15 visemes. You can find more details here how to use Oculus Lipsync with Avatar SDK: Oculus LipSync Sample
• mobile_51: Set of 51 facial blendshapes. This set is compatible with Apple ARKit: https://developer.apple.com/documentation/arkit/creating_face-based_ar_experiences (available for HEAD 2.0 pipeline)
• visemes_15: Set of 14 visemes (PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou) and one additional blendshape: Laughter. This set is compatible with Oculus Lipsync visemes. You can find more details here on how to use Oculus Lipsync with Avatar SDK: Oculus LipSync Sample

Model Info

This group of flags configures which meta-information should be generated for the avatar.

• Hair Color: Compute average haircut color from the submitted photo.
• Skin Color: Compute average skin color from the submitted photo.
• Gender: Predict a person's gender from the submitted photo (male or female) and confidence.
• Age: Classify to which age group the person from the photo belongs to. There are two groups: Child and Adult.
• Facial Landmarks: Compute facial landmarks.
• Eye Sclera Color: Compute average eye sclera color from the submitted photo.
• Eye Iris Color: Compute average eye iris color from the submitted photo.
• Lips Color: Compute average lips color from the submitted photo.
• Predict Haircut: Predict which haircut matches best to submitted photo. The predicted haircut is chosen from the available haircuts set, not from the selected.
• Race: Classify to which race the person from the photo belongs to. Possible values are: black, asian and white. The result of prediction is represented as a set of confidence values for possible values.

Colors correspondance:

Parameters availability:

Parameter Name	Face pipeline	Head 1.2 pipeline	Styled Face pipeline	Head 2.0 (head/mobile)	Head 2.0 (bust/mobile)
Hair Color	Plus	-	Plus	Pro	Pro
Skin Color	Plus	Plus	Plus	Pro	Pro
Gender	Plus	Plus	Plus	Pro	Pro
Age	Plus	Plus	Plus	Pro	Pro
Facial Landmarks	Plus	Plus	Plus	Pro	-
Eye Sclera Color	Plus	Plus	Plus	Pro	Pro
Eye Iris Color	Plus	Plus	Plus	Pro	Pro
Lips Color	Plus	Plus	Plus	-	-
Predict Haircut	Plus	-	Plus	Pro	Pro
Race	-	Plus	-	Pro	Pro

Avatar Modifications

Group of parameters that allows modifying the avatar:

• Allow Modify Neck: By default all avatars have the same neck size for all models for easier attaching to the body. If the flag is set to True, the neck will be modified to better match the submitted photo.
• Curved Bottom: Make the bottom of the Head model slightly curved. Applied only to the Head avatars. See model comparison below.
  
• Slightly Cartoonish Texture: Make the model texture looking with a slightly cartoonish effect. Applied only to the Head avatars.
  
• Parametric eyes texture: Replace eye texture from submitted photo to generated one with sclera and iris colors match the photo version.
  
• Parametric eyes texture v2: Replace eye texture from submitted photo to generated one with sclera and iris colors match the photo version. add_glare and add_eyelid_shadow flags are ignored when this flag is set to true.
  
• Add Glare: Add a glare to the eyes on the model texture. Works only when parametric_eyes_texture is set.
  
• Add Eyelid Shadow: Add an eyelid shadow on the model texture. Works only when parametric_eyes_texture is set.
  
• Eye Iris Color: Allow to set the iris color. Works only when parametric_eyes_texture is set.
  
• Eye Sclera Color: Allow to set the eye sclera color. Works only when parametric_eyes_texture is set.
  
• Lips Color: Allow to set the lips color.
  
• Caricature Amount: Factor to strengthen differences between average model and a particular avatar. Valid values range is [0, +∞). The usable values range is model dependent and usually should be lower than 5.
• Teeth Color: Recolor teeth directly on a model texture.
  
• Hair Color: Recolor generated haircut directly on a haircut texture.
  
• Texture Size: Size of the resulting model's texture.
• Generated haircut texture size: Size of generated haircut's texture.
• Generated haircut faces count: Number of faces in the generated haircut.
  
• Remove Smile: Detect smile on input photo and remove it if detected.
  
• Remove Glasses: Remove glasses on the submitted photo if detected.
  
• Remove Stubble: Attempt to remove a stubble on the source photo before the actual avatar computation start.
  
• Enhance Lighting: Attempt to make lighting more uniform.
  

Parameters availability:

Parameter Name	Face pipeline	Head 1.2 pipeline	Styled Face pipeline	Head 2.0 (head/mobile)	Head 2.0 (bust/mobile)
Allow Modify Neck	Plus	-	Plus	Pro	-
Curved Bottom	-	Plus	-	-	Pro
Slightly Cartoonish Texture	-	Plus	-	Pro	Pro
Parametric Eyes Texture	-	Plus	Always Set	Pro	Pro
Parametric Eyes Texture V2	-	-	-	Pro	Pro
Add Glare	-	-	Plus	Pro	Pro
Add Eyelid Shadow	-	-	Plus	Pro	Pro
Eye Iris Color	-	-	Plus	Pro	Pro
Eye Sclera Color	-	-	Plus	Pro	Pro
Lips Color	Plus	-	Plus	-	-
Caricature Amount	Plus	-	Plus	-	-
Teeth Color	Plus	-	Plus	Pro	Pro
Hair Color	-	-	-	Pro	Pro
Texture Size	-	-	-	Pro	Pro
Generated haircut texture size	-	-	-	Pro	Pro
Generated haircut faces count	-	-	-	Pro	Pro
Remove Smile	-	Plus	-	Pro	Pro
Remove Glasses	-	Plus	-	Pro	Pro
Remove Stubble	-	Plus	-	Pro	Pro
Enhance Lighting	-	Plus	-	Pro	Pro

Shape Modifications

Group of parameters that allows modifying the avatar geometry:

• Cartoonish Level v0.3: Level of the cartoon-like stylization. Applied only to the Styled Face pipeline.
• Cartoonish Level v1.0: Level of the cartoon-like stylization. Applied to the Head 1.2 and Head 2.0 | head/mobile pipelines.

Styled Face, Cartoonish Level v0.3 = 0.75

Head 1.2, Cartoonish Level v1.0 = 0.75

Head 2.0 | bust/mobile, Cartoonish Level v1.0 = 0.5

Additional Textures

The list of the additional textures that can be generated along with the default one:

• slightly_cartoonish_texture: Slightly smoother than regular texture.
  
• smooth_eyelashes_texture: Regular texture with smoothen area of eyelashes.
  
• lips_mask: Mask of the lips on the regular texture.
  
• cartoonish_texture: Uniformly colored texture with cartoon-like effect.
  
• metallic_map: Metallic map
  
• roughness_map: Roughness map
  

Textures availability:

Texture Name	Face pipeline	Head 1.2 pipeline	Styled Face pipeline	Head 2.0 (head/mobile)	Head 2.0 (bust/mobile)
slightly_cartoonish_texture	Plus	-	Plus	-	-
smooth_eyelashes_texture	Plus	-	Plus	-	-
lips_mask	Plus	Plus	Plus	Pro	Pro
cartoonish_texture	-	-	Plus	-	-
metallic_map	-	Plus	-	-	Pro
roughness_map	-	Plus	-	-	Pro

Implementation

See ParametersSample.cs for implementation details.
See also FAQ and the getting started instructions on the main page: Main Page.
Location: Assets/itseez3d/avatar_sdk/samples_cloud/04_parameters_sample_cloud/scenes/04_parameters_sample_cloud.unity.