Guide for running Basaran an open-source alternative to the OpenAI text completion API

Introduction: Basaran is an open-source alternative to the OpenAI text completion API - Repo. It is a powerful tool that can help you generate text based on a given input. In this simple guide, we will show you how to run Basaran on Runpod using a predefined template. As a bonus, we will also teach you how to change the model by editing the MODEL environment variable at pod creation if you want to use a different model than the default bigscience/bloomz-560m.

Step 1: Open the Runpod Template Click on the following link to open the Runpod template for Basaran: https://runpod.io/gsc?template=7ito7h393l&ref=vfker49t

Step 2: Create pod and let it pull the image.

Step 3: Edit the MODEL Environment Variable (Optional) If you want to use a different model than the default bigscience/bloomz-560m, you can change the MODEL environment variable. To do this go to edit pod setting, find the "Environment Variables" section in the pod edit interface. Locate the variable named "MODEL" and replace its value with the identifier of the model you wish to use. For example, you could replace bigscience/bloomz-560m with user/repo.

Note: Make sure to use a valid model identifier that is compatible with Basaran.

Step 4: After you've made any desired changes to the environment variables, reset the pod to apply new model.

Step 5: Access the Basaran Interface Once your pod is running, click on the "Connect via HTTP [Port 80]" button to access the Basaran web interface. From here, you can start using Basaran to generate text completions based on your input.
Note: You can also use API by adding /v1/completions at the end of your proxy URL.

Conclusion: That's it! You've successfully set up and launched Basaran on Runpod using the provided template. Now you can enjoy using this open-source alternative to OpenAI's text completion API. Remember that you can always change the model by editing the MODEL environment variable during pod creation.
Happy text generating!

Hopefully, this blog post has given you a better understanding of how to use the Fast Stable template. If you have any questions or need help, don't hesitate to send message on RunPod's Discord or other communities for support.