Sign in Subscribe

How to create a custom automatic1111 serverless deployment with your own custom model

Zhen Lu

• 5 min read
How to create a custom automatic1111 serverless deployment with your own custom model

Have you ever wanted to create your own serverless automatic1111 endpoint with a custom model of your choice that can scale up and down? Now you can do so without much hassle by following this guide!

Pre-requisites

A computer (local or cloud) with the following:

  • docker installed
  • git installed
  • a fairly fast upload speed
  • at least 100GB of free disk
  • your custom model on the machine, or a public link to get the custom model

We are going to be using the example here as a base: https://github.com/runpod/containers/tree/main/serverless-automatic. You can absolutely write your own custom worker and do whatever you like with it, but we'll try to keep it as dead simple as possible in this tutorial.

Start by navigating to the directory you would like to work in, and then running a git clone on that repository:

git clone https://github.com/runpod/containers.git

This will create a directory called "containers" in your current folder, you can then change to the appropriate example by running

cd containers/serverless-automatic

You should see three files here

- Dockerfile
- handler.py
- start.sh

The Dockerfile is the instruction file that builds your docker image.

The handler file is the python code that is run as your worker.

The start script is the script that is run when your container starts. It is responsible for invoking handler and starting the automatic api internally.

Now, if we want to replace the default model with a custom model, we just need to do a few things. First, we need to delete the old model from the container image, then we need to add our own custom model, and finally we need to point the start script at the new model. All in all, it should require just a few lines of code changes!

If you have your model file locally:

I'll be using the civitai safetensors model from https://civitai.com/models/4823/deliberate in this example. If you've downloaded it locally already, you would edit the Dockerfile to look something like this

FROM runpod/stable-diffusion:web-automatic-1.5.16

SHELL ["/bin/bash", "-c"]

ENV PATH="${PATH}:/workspace/stable-diffusion-webui/venv/bin"

WORKDIR /

RUN rm /workspace/v1-5-pruned-emaonly.ckpt
ADD  model.safetensors /

RUN pip install -U xformers
RUN pip install runpod

ADD handler.py .
ADD start.sh /start.sh
RUN chmod +x /start.sh

CMD [ "/start.sh" ]

Given that you have your model file in the same directory and it is called model.safetensors. The line

ADD  model.safetensors /

will add your model file to the container image.

If you want to download it during the build process from a public link:

Alternatively, if you would rather have docker download the model from the internet, you can use a RUN wget command instead of an ADD command like follows:

FROM runpod/stable-diffusion:web-automatic-1.5.16

SHELL ["/bin/bash", "-c"]

ENV PATH="${PATH}:/workspace/stable-diffusion-webui/venv/bin"

WORKDIR /

RUN rm /workspace/v1-5-pruned-emaonly.ckpt
RUN wget -O model.safetensors https://civitai.com/api/download/models/5616
RUN pip install -U xformers
RUN pip install runpod

ADD handler.py .
ADD start.sh /start.sh
RUN chmod +x /start.sh

CMD [ "/start.sh" ]

Now set the model using the start flags in automatic:

The only other thing to do is to edit the start script to point to the correct model file. In this case, we have stored our model as /model.safetensors, so we will edit it accordingly to look like this:

#!/bin/bash
echo "Container Started"
export PYTHONUNBUFFERED=1
source /workspace/venv/bin/activate
cd /workspace/stable-diffusion-webui
echo "starting api"
python webui.py --port 3000 --nowebui --api --xformers --ckpt /model.safetensors &
cd /

echo "starting worker"
python -u handler.py

At this point, you are ready to build your docker image. You can do so by running a command like this:

sudo DOCKER_BUILDKIT=1 docker build .

After you have built your image, you can push it to your favorite container registry. For docker hub you can do the following:

Getting an image to Docker Hub

Imagine you made your own Docker image and would like to share it with the world you can sign up for an account on https://hub.docker.com/. After verifying your email you are ready to go and upload your first docker image.

  1. Log in on https://hub.docker.com/
  2. Click on Create Repository.
  3. Choose a name (e.g. automatic-custom) and a description for your repository and click Create.
  4. Log into the Docker Hub from the command line
docker login --username=yourhubusername --email=youremail@company.com

just with your own user name and email that you used for the account. Enter your password when prompted. If everything worked you will get a message similar to

WARNING: login credentials saved in /home/username/.docker/config.json
Login Succeeded
  1. Check the image ID using
docker images

and what you will see will be similar to

REPOSITORY              TAG       IMAGE ID           CREATED         SIZE
automatic-custom      1.0.0   023ab91c6291     3 minutes ago     9.975 GB

and tag your image

docker tag bb38976d03cf yourhubusername/automatic-custom:1.0.0

The number must match the image ID and :1.0.0is the tag. In general, a good choice for a tag is something that will help you understand what this container should be used in conjunction with, or what it represents. In this case, it's the first version, so we tag it 1.0.0

  1. Push your image to the repository you created
docker push yourhubusername/automatic-custom:1.0.0

Your image is now available for everyone to use and you can add it to your template like so by creating a new template and filling it out with the container image name you just pushed:

You can then use your template in your API by selecting it from the dropdown menu.

Note: This example is using SD version 1.5 models. If you want to use SD v2.1 based models, you should use the replace

FROM runpod/stable-diffusion:web-automatic-1.5.15

in the dockerfile with the 2.1.x version and delete the standard v2.1 model accordingly.

At this point, you can follow the documentation for how to deploy your API and use it:

https://docs.runpod.io/serverless-ai/custom-apis/autoscaling

https://docs.runpod.io/serverless-ai/custom-apis/using-your-api

Final Thoughts

While using the automatic1111 API is convenient because it has a lot of built-in functionality, there are a few things to consider.

Firstly, the cold start time for this API is about 20 seconds, vs 10 seconds for a raw diffusers-based worker. Automatic1111 does take a bit of time to go through its internal checks as well as starting the uvicorn API that they use internally.

Secondly, because the API has so much functionality, it's very difficult to make sure that every piece of it works, so be aware!

Other than that, it's a really great way to get started without having to define a bunch of your own standards or rolling your own code base. Happy hunting!

Sign up for more like this.

Enter your email
Subscribe