Scene #05: Parameters Sample

This scene demonstrates API that allows working with computation parameters and flags. These parameters and flags configure which auxiliary features and resources should be generated for the avatar.

The SDK will tell you which resources and parameters are available for your account (this is the full list that you see in the UI). Then before the avatar is generated you can specify which resources you need and which parameters you would like to apply.

Detailed information about parameters can be found in our Web Api Documentation.

In this sample the Styled Face pipeline is used. It is the subtype (indie/legacy_styled) of the Animated Face pipeline. Main purpose of the Styled Face is generation of "cartoonish" avatars.

Currently, the following groups with parameters are available:

• Haircuts
• Blendshapes
• Model Info
• Avatar Modifications
• Shape Modifications
• Additional Textures

Code Usage

ComputationParameters is used to specify required parameters. It is passed to the IAvatarProvider.InitializeAvatarAsync method during avatar initialization.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, PipelineType pipeline)
{
    ComputationParameters computationParameters = ComputationParameters.Empty;

    computationParameters.modelInfo = new ModelInfoGroup();
    computationParameters.modelInfo.age.Value = true;
    computationParameters.modelInfo.skinColor.Value = true;

    computationParameters.avatarModifications = new AvatarModificationsGroup();
    computationParameters.avatarModifications.removeGlasses.Value = true;
    computationParameters.avatarModifications.removeSmile.Value = true;
    computationParameters.avatarModifications.enhanceLighting.Value = true;

    var initializeRequest = avatarProvider.InitializeAvatarAsync(photoBytes, string.Empty, null, pipeline, computationParameters);
    yield return initializeRequest;
}

Use IAvatarProvider.GetParametersAsync method to get all available parameters for the given pipelne.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, PipelineType pipeline)
{
    var parametersRequest = avatarProvider.GetParametersAsync(ComputationParametersSubset.ALL, pipeline);
    yield return parametersRequest;

    ComputationParameters availableParameters = parametersRequest.Result;

    ComputationParameters specifiedParameters = ComputationParameters.Empty;

    specifiedParameters.modelInfo = new ModelInfoGroup();
    if (availableParameters.modelInfo.predictHaircut.IsAvailable)
        specifiedParameters.modelInfo.predictHaircut.Value = true;
    if (availableParameters.modelInfo.hairColor.IsAvailable)
        specifiedParameters.modelInfo.hairColor.Value = true;

    specifiedParameters.avatarModifications = new AvatarModificationsGroup();
    if (availableParameters.avatarModifications.removeGlasses.IsAvailable)
        specifiedParameters.avatarModifications.removeGlasses.Value = true;
    if (availableParameters.avatarModifications.removeSmile.IsAvailable)
        specifiedParameters.avatarModifications.removeSmile.Value = true;
    if (availableParameters.avatarModifications.enhanceLighting.IsAvailable)
        specifiedParameters.avatarModifications.enhanceLighting.Value = true;

    specifiedParameters.blendshapes = availableParameters.blendshapes;
    specifiedParameters.haircuts = availableParameters.haircuts;
    specifiedParameters.additionalTextures = availableParameters.additionalTextures;

    var initializeRequest = avatarProvider.InitializeAvatarAsync(photoBytes, string.Empty, null, pipeline, specifiedParameters);
    yield return initializeRequest;
}

Parameters

Haircuts

There are two sets of artificial haircuts:

• Base set: 3 male and 3 female haircuts
• Facegen set: 39 haircuts in total, some of them are unisex

It is possible to add custom haircuts to your account. Please contact us at suppo.nosp@m.rt@a.nosp@m.vatar.nosp@m.sdk..nosp@m.com to implement this, we will provide the implementation details on a custom basis. Once the custom haircuts are added, the SDK will return more haircuts in the available haircuts list.

Blendshapes

Avatar SDK supports animated avatars using blendshapes. There are two different blendshapes sets available in the Avatar SDK Unity plugin which should be used depending on your use-case: general animation or lip-sync. E.g. if you want to use lipsync functionality (such as Oculus Lipsync SDK) with your avatar you may want to generate additional visemes blendshapes, but if you don't use lipsync you should tell the SDK to skip these blendshapes to save calculation and download time.

Animated Face blendshapes:

• legacy_45: Basic set of 45 facial blendshapes.
• visems_17: Set of 15 visemes (aa, ch, ee, ih, oh, ou, th, dd, ff, kk, nn, pp, rr, sil, ss) and two additional blendshapes: frown and smile. This set is compatible with Oculus Lipsync that required these 15 visemes.

Head 1.2 blendshapes:

• mobile_51: Set of 51 facial blendshapes. This set is compatible with Apple ARKit: https://developer.apple.com/documentation/arkit/creating_face-based_ar_experiences
• visems_15: Set of 14 visemes (PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou) and one additional blendshape: Laughter. This set is compatible with Oculus Lipsync visemes.

Model Info

This group of flags configures which meta-information should be generated for the avatar.

• Hair Color: Compute average haircut color from submitted photo.
• Skin Color: Compute average skin color from submitted photo.
• Gender: Predict person's gender from submitted photo (male or female) and confidence.
• Age: Classify to which age group the person from the photo belongs to. There are two groups: Child and Adult.
• Facial Landmarks: Compute facial landmarks.
• Eye Sclera Color: Compute average eye sclera color from submitted photo.
• Eye Iris Color: Compute average eye iris color from submitted photo.
• Lips Color: Compute average lips color from submitted photo.
• Predict Haircut: Predict which haircut matches best to submitted photo. Predicted haircut is chosen from the available haircuts set, not from the selected.
• Predict Race: Classify to which race the person from the photo belongs to. Possible values are: black, asian and white. Result of prediction is represented as set of confidence values for possible values.

Colors correspondance:

Parameters availability:

Parameter Name	Animated Face pipeline	Styled Face pipeline	Head 1.2 pipeline
Hair Color	+	+	-
Skin Color	+	+	+
Gender	+	+	+
Age	+	+	+
Facial Landmarks	+	+	+
Eye Sclera Color	+	+	+
Eye Iris Color	+	+	+
Lips Color	+	+	+
Predict Haircut	+	+	-
Predict Race	+	+	+

Avatar Modifications

Group of parameters that allows to modify the avatar:

• Allow Modify Neck: By default all avatars have the same neck size for all models for easier attaching to the body. If flag is set to True, neck will be modified to better match submitted photo.
• Curved Bottom: Make bottom of the Head model slightly curved. Applied only to the Head avatars. See models comparison below.
  
• Slightly Cartoonish Texture: Make the regular texture looking with slightly cartoonish effect. Applied only to the Head avatars.
  
• Parametric Eyes Texture: Replace eye texture from submitted photo to generated one with sclera and iris colors match the photo version.
  
• Add Glare: Add a glare to the eyes on the regular texture. Works only when Parametric Eyes Texture is set.
  
• Add Eyelid Shadow: Add an eyelid shadow on the regular texture. Works only when Parametric Eyes Texture is set.
  
• Eye Iris Color: Allow to set the iris color. Works only when Parametric Eyes Texture is set.
  
• Eye Sclera Color: Allow to set the eye sclera color. Works only when Parametric Eyes Texture is set.
  
• Lips Color: Allow to set the lips color.
• Teeth Color: Recolor teeth directly on a model texture.
• Caricature Amount: Factor to strengthen differences between average model and particular avatar. Valid values range is [0, +∞). Usable values range is model dependent and usually should be lower than 5.
• Repack Texture: Optimize texture layout to occupy less space. See textures comparison below:
  
• Remove Smile: Detect smile on input photo and remove if detected.
  
• Remove Glasses: Attempt to remove glasses on the submitted photo before the actual avatar computation start.
  
• Enhance Lighting: Attempt to make lighting more uniform.
  

Parameters availability:

Parameter Name	Animated Face pipeline	Styled Face pipeline	Head 1.2 pipeline
Allow Modify Neck	+	+	-
Curved Bottom	-	-	+
Slightly Cartoonish Texture	-	-	+
Parametric Eyes	-	Always Set	+
Add Glare	-	+	+
Add Eyelid Shadow	-	+	+
Eye Iris Color	-	+	+
Eye Sclera Color	-	+	+
Lips Color	+	+	+
Teeth Color	+	+	+
Caricature Amount	+	+	-
Repack Texture	-	-	+
Remove smile	-	-	+
Remove Glasses	-	-	+
Enhance Lighting	-	-	+

Shape Modifications

Group of parameters that allows to modify the avatar geometry:

• Cartoonish Level: Level of the cartoon-like stylization. Applied only to the "Styled Face" pipeline.

Additional Textures

The list of the additional textures that can be generated along with the default one:

• slightly_cartoonish_texture: Slightly smoother than regular texture.
  
• smooth_eyelashes_texture: Regular texture with smoothen area of eyelashes.
  
• lips_mask: Mask of the lips on the regular texture.
  
• cartoonish_texture: Uniformly colored texture with cartoon-like effect.
  
• metallic_map: Metallic map.
  
• roughness_map: Roughness map.
  

Textures availability:

Texture Name	Amimated Face pipeline	Styled Face pipeline	Head 1.2 pipeline
slightly_cartoonish_texture	+	+	-
smooth_eyelashes_texture	+	+	-
lips_mask	+	+	+
cartoonish_texture	-	+	-
metallic_map	-	-	+
roughness_map	-	-	+

Implementation

See ParametersSample.cs for implementation details.

See also FAQ and the getting started instructions on the main page: Main Page.