ControlNet with Civitai

Stable Diffusion with ControlNet

This widget supports multiple actions. For a more comprehensive understanding of its functionality, we recommend reviewing the following documentation carefully.

You need to pass both the action and other input parameters of the chosen action to your module_config

Try it in the Widget Center

Click this url to try this widget and copy the Pro Config template.

Usage

Generate Picture with ControlNet

action txt2img

Input Parameters

Name	Type	Description	Default	Required
action	string	The action of ControlNet, txt2img or img2img	txt2img
model	string	The model id from civitai (SD1.5, SDXL 1.0, PlaygroundV2). How to get it? Click on a model page on civitai, and copy the series number within the download link.	64094
controlnet_model	string	The ControlNet model id from civitai. How to get it? Click on a model page on civitai, and copy the series number within the download link.	10971
image	string	The input image, can be a url or base64 sting
no_mask	boolean	Whether to use mask	False
mask	string	The correspond mask, can be a url or base64 sting. 1 for mask region
prompt	string	The text prompt for ControlNet. Add lora? add `` to your prompt. `$id` is the series number and `$weight` is the lora weight you want (always set to 1.0). You can use multiple loras.
negative_prompt	string	The negative prompt for ControlNet.	(worst quality, low quality:1.4),(malformed hands:1.4),(poorly drawn hands:1.4),(mutated fingers:1.4),(extra limbs:1.35),(poorly drawn face:1.4),bad leg,strange leg, poor eyes, full screen of face
controlnet_module	string	The ControNet module	none
guidance_start	number	ControlNet guidance start	0
guidance_end	number	ControlNet guidance end	1
control_mode	string	The improved guess mode	Balanced
weight	number	The weight of the controlnet model	1
resize_mode	string	Four modes for output shape calculation: (1) Keep: keep original shape, (2) Certain: based on input width/length (divisible by 32), (3,4) min/max ratio: keep aspect ratio, the resize factor is the min/max of (h/H, w/W).	certain
threshold_a	integer	The threshold A for controlnet model	64
threshold_b	integer	The threshold B for controlnet model	64
steps	integer	Steps for sampler to step whle sampling	25
cfg_scale	number	Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results. Default to 7.	7.0
sampler	string	Sampler for diffusion model inference	DPM++ 2M
height	integer	Height of the generated images	512
width	integer	Width of the generated images	512
seed	integer	Random seed for generation process. -1 means random seed	-1
clip_skip	integer	Early stopping parameter for CLIP model; 1 is stop at last layer as usual, 2 is stop at penultimate layer, etc.	1

Output Parameters

Name	Type	Description	File Type
url	string	The url of generated image, stored in the cloud. Only temporarily effective, will be cleared in a few hours.	image

Output Example

success

{
  "url": "https://image.myshell.ai/image/chat/embed_obj/7758545/202405220201/6ee8737899794e9bb2e12b5844a4b11a.jpg"
}

Inpaint with ControlNet

action img2img

Input Parameters

Name	Type	Description	Default	Required
action	string	The action of ControlNet, txt2img or img2img	txt2img
model	string	The model id from civitai (SD1.5, SDXL 1.0, PlaygroundV2). How to get it? Click on a model page on civitai, and copy the series number within the download link.	64094
controlnet_model	string	The ControlNet model id from civitai. How to get it? Click on a model page on civitai, and copy the series number within the download link.	10971
image	string	The input image, can be a url or base64 sting
no_mask	boolean	Whether to use mask	False
mask	string	The correspond mask, can be a url or base64 sting. 1 for mask region
prompt	string	The text prompt for ControlNet. Add lora? add `` to your prompt. `$id` is the series number and `$weight` is the lora weight you want (always set to 1.0). You can use multiple loras.
negative_prompt	string	The negative prompt for ControlNet.	(worst quality, low quality:1.4),(malformed hands:1.4),(poorly drawn hands:1.4),(mutated fingers:1.4),(extra limbs:1.35),(poorly drawn face:1.4),bad leg,strange leg, poor eyes, full screen of face
controlnet_module	string	The ControNet module	none
guidance_start	number	ControlNet guidance start	0
guidance_end	number	ControlNet guidance end	1
control_mode	string	The improved guess mode	Balanced
weight	number	The weight of the controlnet model	1
resize_mode	string	Four modes for output shape calculation: (1) Keep: keep original shape, (2) Certain: based on input width/length (divisible by 32), (3,4) min/max ratio: keep aspect ratio, the resize factor is the min/max of (h/H, w/W).	certain
threshold_a	integer	The threshold A for controlnet model	64
threshold_b	integer	The threshold B for controlnet model	64
steps	integer	Steps for sampler to step whle sampling	25
cfg_scale	number	Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results. Default to 7.	7.0
sampler	string	Sampler for diffusion model inference	DPM++ 2M
height	integer	Height of the generated images	512
width	integer	Width of the generated images	512
seed	integer	Random seed for generation process. -1 means random seed	-1
clip_skip	integer	Early stopping parameter for CLIP model; 1 is stop at last layer as usual, 2 is stop at penultimate layer, etc.	1
mask_blur	integer	Mask blur refers to the feathering of a mask (from edges to inside the mask), adjusted between 0-64. A smaller value results in sharper edges. Default to 4	4
inpainting_fill	integer	Choose the fill content in mask: 0 - fill, 1 - original, 2 - latent noise, 3 - latent nothing	1
inpainting_mask_invert	integer	0 - Inpaint masked region, 1 - Inpaint not masked region	0
denoising_strength	number	Strength of image transfomation during inpainting precess. High means more influence during transformation	0.7

Output Parameters

Name	Type	Description	File Type
url	string	The url of generated image, stored in the cloud. Only temporarily effective, will be cleared in a few hours.	image

Output Example

success

{
  "url": "https://image.myshell.ai/image/chat/embed_obj/7758545/202405221652/f9550811186e4152952020ea097375d6.jpg"
}