Scene #05: Parameters Sample

This scene demonstrates API that allows working with computation parameters and flags. These parameters and flags configure which auxiliary features and resources should be generated for the avatar.

The SDK will tell you which resources and parameters are available for your account (this is the full list that you see in the UI). Then before the avatar is generated you can specify which resources you need and which parameters you would like to apply.

Currently, the following groups with parameters are available:

• Haircuts
• Blendshapes
• Model Info
• Avatar Modifications
• Shape Modifications
• Additional Textures

Code Usage

ComputationParameters is used to specify required parameters. It is passed to the IAvatarProvider.InitializeAvatarAsync method during avatar initialization.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, PipelineType pipeline)
{
    ComputationParameters computationParameters = ComputationParameters.Empty;

    computationParameters.modelInfo = new ModelInfoGroup();
    computationParameters.modelInfo.age.Value = true;
    computationParameters.modelInfo.skinColor.Value = true;

    computationParameters.avatarModifications = new AvatarModificationsGroup();
    computationParameters.avatarModifications.removeGlasses.Value = true;
    computationParameters.avatarModifications.removeSmile.Value = true;
    computationParameters.avatarModifications.enhanceLighting.Value = true;

    var initializeRequest = avatarProvider.InitializeAvatarAsync(photoBytes, string.Empty, null, pipeline, computationParameters);
    yield return initializeRequest;
}

Use IAvatarProvider.GetParametersAsync method to get all available parameters for the given pipelne.

IEnumerator ConfigureComputationParameters(byte[] photoBytes, PipelineType pipeline)
{
    var parametersRequest = avatarProvider.GetParametersAsync(ComputationParametersSubset.ALL, pipeline);
    yield return parametersRequest;

    ComputationParameters availableParameters = parametersRequest.Result;

    ComputationParameters specifiedParameters = ComputationParameters.Empty;

    specifiedParameters.modelInfo = new ModelInfoGroup();
    if (availableParameters.modelInfo.predictHaircut.IsAvailable)
        specifiedParameters.modelInfo.predictHaircut.Value = true;
    if (availableParameters.modelInfo.hairColor.IsAvailable)
        specifiedParameters.modelInfo.hairColor.Value = true;

    specifiedParameters.avatarModifications = new AvatarModificationsGroup();
    if (availableParameters.avatarModifications.removeGlasses.IsAvailable)
        specifiedParameters.avatarModifications.removeGlasses.Value = true;
    if (availableParameters.avatarModifications.removeSmile.IsAvailable)
        specifiedParameters.avatarModifications.removeSmile.Value = true;
    if (availableParameters.avatarModifications.enhanceLighting.IsAvailable)
        specifiedParameters.avatarModifications.enhanceLighting.Value = true;

    specifiedParameters.blendshapes = availableParameters.blendshapes;
    specifiedParameters.haircuts = availableParameters.haircuts;

    var initializeRequest = avatarProvider.InitializeAvatarAsync(photoBytes, string.Empty, null, pipeline, specifiedParameters);
    yield return initializeRequest;
}

Parameters

Haircuts

Unity plugin includes the following sets of artificial haircuts:

• Base set: 3 male and 3 female haircuts
• Plus set: set of haircuts available for the HEAD 2.0 pipeline
• Generated: haircut generated from original image.

It is possible to add custom haircuts to your account. Please contact us at suppo.nosp@m.rt@a.nosp@m.vatar.nosp@m.sdk..nosp@m.com to implement this, we will provide the implementation details on a custom basis. Once the custom haircuts are added, the SDK will return more haircuts in the available haircuts list.

Blendshapes

Avatar SDK supports avatars animation using blendshapes. There are two different blendshapes sets available in the Avatar SDK Local Compute Unity plugin which should be used depending on your use-case: general animation and lip-sync. E.g. if you want to use lipsync functionality (such as Oculus Lipsync SDK) with your avatar you may want to generate additional visemes blendshapes, but if you don't use lipsync you should tell the SDK to skip these blendshapes to save calculation time.

• mobile_51: Set of 51 facial blendshapes. This set is compatible with Apple ARKit: https://developer.apple.com/documentation/arkit/creating_face-based_ar_experiences
• visems_15: Set of 14 visemes (PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou) and one additional blendshape: Laughter. This set is compatible with Oculus Lipsync visemes. You can find more details here on how to use Oculus Lipsync with Avatar SDK: Oculus LipSync Sample

Model Info

This group of flags configures which meta-information should be generated for the avatar.

• Hair Color: Compute average haircut color from submitted photo.
• Skin Color: Compute average skin color from submitted photo.
• Gender: Predict person's gender from submitted photo (male or female) and confidence.
• Age: Classify to which age group the person from the photo belongs to. There are two groups: Child and Adult.
• Eye Sclera Color: Compute average eye sclera color from submitted photo.
• Eye Iris Color: Compute average eye iris color from submitted photo.
• Facial Landmarks: Compute facial landmarks. Available for the head/mobile pipeline only.
• Race: Classify to which race the person from the photo belongs to. Possible values are: black, asian and white. The result of prediction is represented as a set of confidence values for possible values.

Colors correspondance:

Avatar Modifications

Group of parameters that allows to modify the avatar:

• Allow Modify Neck: By default this flag is set to True and the neck is modified to better match submitted photo. Disable it if you need the fixed neck size for all models for easier attaching to the body. Applicable only for the "head/mobile" subtype.
• Curved Bottom: Make the bottom of the avatar model slightly curved. Applied only to the "bust/mobile" avatars. See models comparison below.
  
• Slightly Cartoonish Texture: Make the model texture looking with a slightly cartoonish effect.
  
• Parametric eyes texture: Replace eye texture from submitted photo to generated one with sclera and iris colors match the photo version.
• Parametric eyes texture v2: Replace eye texture from submitted photo to generated one with sclera and iris colors match the photo version. add_glare and add_eyelid_shadow parameters are ignored with the parametric_eyes_v2.
  
• Add Eyelid Shadow: Add an eyelid shadow on the model texture. Works only when parametric_eyes_texture is set.
  
• Add Glare: Add a glare to the eyes on the model texture. Works only when parametric_eyes_texture is set.
  
• Eye Iris Color: Allow to set the iris color. Works only when parametric_eyes_texture is set.
  
• Eye Sclera Color: Allow to set the eye sclera color. Works only when parametric_eyes_texture is set.
  
• Hair color: Recolor generated haircut directly on a haircut texture.
  
• Teeth Color: Recolor teeth directly on a model texture.
  
• Generated haircut texture size: Size of generated haircut's texture.
• Generated haircut faces count: Number of faces in the generated haircut.
  
• Texture Size: Size of the resulting model's texture.
• Remove Smile: Detect smile on input photo and remove it if detected.
  
• Remove Glasses: Remove glasses on the submitted photo if detected.
  
• Enhance Lighting: Attempt to make lighting more uniform.
  

Shape Modifications

Group of parameters that allows modifying the avatar geometry:

• Cartoonish Level v1.0: Level of the cartoon-like stylization. Applied only to the "Head 2.0 | head/mobile" pipeline.
  

  Cartoonish Level v1.0 = 0.5

Additional Textures

The list of the additional textures that can be generated along with the default one:

• lips_mask: Mask of the lips on the regular texture.
• metallic_map: Metallic map
• roughness_map: Roughness map
• normal_map: Normal map

The PBR textures (metallic_map, roughness_map, normal_map) works better in the Linear color space. So we recommend to change the color space from Gamma to Linear before using them (Build Settings -> Player Settings -> Other -> Color Space).

Implementation

See ParametersSample.cs for implementation details.
See also FAQ and the getting started instructions on the main page: Main Page.