2023-12-04AIGC-StableDiffusion和SadTalker-搭建及使用

摘要:

2023-12-04 AIGC-SadTalker-搭建及使用

代码仓库:

github /Stability-AI/stablediffusion

github /camenduru/stable-diffusion-webui-colab

github /OpenTalker/SadTalker

github /adofsauron/SadTalker-Video-Lip-Sync

文档:

Stable Diffusion一键安装包 Windows版 - Stable Diffusion中文网

类似D-ID的免费开源虚拟数字人制作工具SadTalker搭建教程及效果演示 - 哔哩哔哩

SadTalker调参实验 - 知乎

github /OpenTalker/SadTalker/blob/main/docs/best_practice.md

SadTalker参数:

(sadtalker) PS D:\sd\SadTalker> python inference.py --help usage: inference.py [-h] [--driven_audio DRIVEN_AUDIO] [--source_image SOURCE_IMAGE] [--ref_eyeblink REF_EYEBLINK] [--ref_pose REF_POSE] [--checkpoint_dir CHECKPOINT_DIR] [--result_dir RESULT_DIR] [--pose_style POSE_STYLE] [--batch_size BATCH_SIZE] [--size SIZE] [--expression_scale EXPRESSION_SCALE] [--input_yaw INPUT_YAW [INPUT_YAW ...]] [--input_pitch INPUT_PITCH [INPUT_PITCH ...]] [--input_roll INPUT_ROLL [INPUT_ROLL ...]] [--enhancer ENHANCER] [--background_enhancer BACKGROUND_ENHANCER] [--cpu] [--face3dvis] [--still] [--preprocess {crop,extcrop,resize,full,extfull}] [--verbose] [--old_version] [--net_recon {resnet18,resnet34,resnet50}] [--init_path INIT_PATH] [--use_last_fc USE_LAST_FC] [--bfm_folder BFM_FOLDER] [--bfm_model BFM_MODEL] [--focal FOCAL] [--center CENTER] [--camera_d CAMERA_D] [--z_near Z_NEAR] [--z_far Z_FAR] optional arguments: -h, --help show this help message and exit --driven_audio DRIVEN_AUDIO path to driven audio --source_image SOURCE_IMAGE path to source image --ref_eyeblink REF_EYEBLINK path to reference video providing eye blinking --ref_pose REF_POSE path to reference video providing pose --checkpoint_dir CHECKPOINT_DIR path to output --result_dir RESULT_DIR path to output --pose_style POSE_STYLE input pose style from [0, 46) --batch_size BATCH_SIZE the batch size of facerender --size SIZE the image size of the facerender --expression_scale EXPRESSION_SCALE the batch size of facerender --input_yaw INPUT_YAW [INPUT_YAW ...] the input yaw degree of the user --input_pitch INPUT_PITCH [INPUT_PITCH ...] the input pitch degree of the user --input_roll INPUT_ROLL [INPUT_ROLL ...] the input roll degree of the user --enhancer ENHANCER Face enhancer, [gfpgan, RestoreFormer] --background_enhancer BACKGROUND_ENHANCER background enhancer, [realesrgan] --cpu --face3dvis generate 3d face and 3d landmarks --still can crop back to the original videos for the full body aniamtion --preprocess {crop,extcrop,resize,full,extfull} how to preprocess the images --verbose saving the intermedia output or not --old_version use the pth other than safetensor version --net_recon {resnet18,resnet34,resnet50} useless --init_path INIT_PATH Useless --use_last_fc USE_LAST_FC zero initialize the last fc --bfm_folder BFM_FOLDER --bfm_model BFM_MODEL bfm model --focal FOCAL --center CENTER --camera_d CAMERA_D --z_near Z_NEAR --z_far Z_FAR