ControlNet with Civitai

Stable Diffusion with ControlNet

This widget supports multiple actions. For a more comprehensive understanding of its functionality, we recommend reviewing the following documentation carefully.

You need to pass both the action and other input parameters of the chosen action to your module_config

Try it in the Widget Center

Click this url to try this widget and copy the Pro Config template.

Usage

Generate Picture with ControlNet

action txt2img

Input Parameters

NameTypeDescriptionDefaultRequired

action

string

The action of ControlNet, txt2img or img2img

txt2img

model

string

The model id from civitai (SD1.5, SDXL 1.0, PlaygroundV2). How to get it? Click on a model page on civitai, and copy the series number within the download link.

64094

controlnet_model

string

The ControlNet model id from civitai. How to get it? Click on a model page on civitai, and copy the series number within the download link.

10971

image

string

The input image, can be a url or base64 sting

no_mask

boolean

Whether to use mask

False

mask

string

The correspond mask, can be a url or base64 sting. 1 for mask region

prompt

string

The text prompt for ControlNet. Add lora? add `` to your prompt. `$id` is the series number and `$weight` is the lora weight you want (always set to 1.0). You can use multiple loras.

negative_prompt

string

The negative prompt for ControlNet.

(worst quality, low quality:1.4),(malformed hands:1.4),(poorly drawn hands:1.4),(mutated fingers:1.4),(extra limbs:1.35),(poorly drawn face:1.4),bad leg,strange leg, poor eyes, full screen of face

controlnet_module

string

The ControNet module

none

guidance_start

number

ControlNet guidance start

0

guidance_end

number

ControlNet guidance end

1

control_mode

string

The improved guess mode

Balanced

weight

number

The weight of the controlnet model

1

resize_mode

string

Four modes for output shape calculation: (1) Keep: keep original shape, (2) Certain: based on input width/length (divisible by 32), (3,4) min/max ratio: keep aspect ratio, the resize factor is the min/max of (h/H, w/W).

certain

threshold_a

integer

The threshold A for controlnet model

64

threshold_b

integer

The threshold B for controlnet model

64

steps

integer

Steps for sampler to step whle sampling

25

cfg_scale

number

Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results. Default to 7.

7.0

sampler

string

Sampler for diffusion model inference

DPM++ 2M

height

integer

Height of the generated images

512

width

integer

Width of the generated images

512

seed

integer

Random seed for generation process. -1 means random seed

-1

clip_skip

integer

Early stopping parameter for CLIP model; 1 is stop at last layer as usual, 2 is stop at penultimate layer, etc.

1

Output Parameters

NameTypeDescriptionFile Type

url

string

The url of generated image, stored in the cloud. Only temporarily effective, will be cleared in a few hours.

image

Output Example

{
  "url": "https://image.myshell.ai/image/chat/embed_obj/7758545/202405220201/6ee8737899794e9bb2e12b5844a4b11a.jpg"
}

Inpaint with ControlNet

action img2img

Input Parameters

NameTypeDescriptionDefaultRequired

action

string

The action of ControlNet, txt2img or img2img

txt2img

model

string

The model id from civitai (SD1.5, SDXL 1.0, PlaygroundV2). How to get it? Click on a model page on civitai, and copy the series number within the download link.

64094

controlnet_model

string

The ControlNet model id from civitai. How to get it? Click on a model page on civitai, and copy the series number within the download link.

10971

image

string

The input image, can be a url or base64 sting

no_mask

boolean

Whether to use mask

False

mask

string

The correspond mask, can be a url or base64 sting. 1 for mask region

prompt

string

The text prompt for ControlNet. Add lora? add `` to your prompt. `$id` is the series number and `$weight` is the lora weight you want (always set to 1.0). You can use multiple loras.

negative_prompt

string

The negative prompt for ControlNet.

(worst quality, low quality:1.4),(malformed hands:1.4),(poorly drawn hands:1.4),(mutated fingers:1.4),(extra limbs:1.35),(poorly drawn face:1.4),bad leg,strange leg, poor eyes, full screen of face

controlnet_module

string

The ControNet module

none

guidance_start

number

ControlNet guidance start

0

guidance_end

number

ControlNet guidance end

1

control_mode

string

The improved guess mode

Balanced

weight

number

The weight of the controlnet model

1

resize_mode

string

Four modes for output shape calculation: (1) Keep: keep original shape, (2) Certain: based on input width/length (divisible by 32), (3,4) min/max ratio: keep aspect ratio, the resize factor is the min/max of (h/H, w/W).

certain

threshold_a

integer

The threshold A for controlnet model

64

threshold_b

integer

The threshold B for controlnet model

64

steps

integer

Steps for sampler to step whle sampling

25

cfg_scale

number

Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results. Default to 7.

7.0

sampler

string

Sampler for diffusion model inference

DPM++ 2M

height

integer

Height of the generated images

512

width

integer

Width of the generated images

512

seed

integer

Random seed for generation process. -1 means random seed

-1

clip_skip

integer

Early stopping parameter for CLIP model; 1 is stop at last layer as usual, 2 is stop at penultimate layer, etc.

1

mask_blur

integer

Mask blur refers to the feathering of a mask (from edges to inside the mask), adjusted between 0-64. A smaller value results in sharper edges. Default to 4

4

inpainting_fill

integer

Choose the fill content in mask: 0 - fill, 1 - original, 2 - latent noise, 3 - latent nothing

1

inpainting_mask_invert

integer

0 - Inpaint masked region, 1 - Inpaint not masked region

0

denoising_strength

number

Strength of image transfomation during inpainting precess. High means more influence during transformation

0.7

Output Parameters

NameTypeDescriptionFile Type

url

string

The url of generated image, stored in the cloud. Only temporarily effective, will be cleared in a few hours.

image

Output Example

{
  "url": "https://image.myshell.ai/image/chat/embed_obj/7758545/202405221652/f9550811186e4152952020ea097375d6.jpg"
}

Last updated