Building a Private Assistant with OpenWebUI on the Frontend and DeepInfra on the Backend

Over the weekend, I created a private assistant with a sophisticated frontend using OpenWebUI (https://openwebui.com/ ) and a backend using DeepInfra (https://deepinfra.com/ ). DeepInfra offers several models and allows you to develop tools using open-source models, even if you have limited GPU resources. The instructions below are super simple and anyone with basic Docker knowledge should be able to set this up.

Setting up DeepInfra

DeepInfra is a powerful platform that simplifies the process of running machine learning models in the cloud. By abstracting away the complexities of server management, GPU allocation, and scaling, DeepInfra allows you to focus on your application and business needs. In this section, we'll walk through the steps to set up DeepInfra, choose a model, and generate an API key. By the end, you'll have a robust backend ready for your assistant.

What is DeepInfra?

DeepInfra is a serverless solution that makes it easy to run the latest machine learning models in the cloud. Here are some key features:

• Serverless Solutions : DeepInfra abstracts away the complexities of managing servers, GPUs, and scaling, allowing you to concentrate on your application.

• Access to Many Models : DeepInfra provides access to a wide range of models, including text and image generation models.

• Cost-Effective : I chose DeepInfra by using a model comparison tool (https://whatllm.vercel.app/ ) to find the cheapest solution for the model I was targeting, Qwen 2.5 72B. Running such a large model on my 3060 with 12GB of VRAM would have been impossible, but services like DeepInfra make it feasible. 

• Security : DeepInfra is secure and does not store any of your prompts or data you send. All data is deleted after inference, ensuring your privacy.

Setting up DeepInfra

Now onto the good parts. Let’s set up DeepInfra with an OpenAI Endpoint for easy integration with OpenWebUI. 

1. Sign Up :

   • Go to the DeepInfra website and sign up for an account: DeepInfra Sign Up .

2. Deploy a New Model :

   • Once you're logged in, navigate to the dashboard: DeepInfra Dashboard .

   • Click on "New Deployment" to start the process.

3. Search for Your Model :

   • In the search bar, type the name of the model you want to use. For my project, I chose Qwen 2.5 70B, but you can select any model that suits your needs.

   • Click on the model to deploy it. Follow the on-screen instructions to configure the model settings.

4. Create an API Key :

   • Go to the API keys section of the dashboard: Create API Key .

   • Click on "New API Key" and follow the prompts to generate a new key.

   • Make sure to save your API key in a secure place. You will need it to authenticate your requests to the model.

5. Enter Your Billing Information:

   • One last thing, before your model can be fully deployed and start processing requests, you need to enter your billing information. Navigate to the billing section of the DeepInfra dashboard and follow the steps to add your payment details.

By following these steps, you'll have your DeepInfra backend set up and ready to integrate with your frontend using OpenWebUI. In the next section go through some quick steps to set up OpenWebUI on your machine.

Setting Up OpenWebUI

Now that your DeepInfra backend is set up and ready to go, it's time to set up the frontend using OpenWebUI. In this section, we'll walk through the steps to install and configure OpenWebUI using Docker to connect with your DeepInfra backend.

What is OpenWebUI?

OpenWebUI is an open-source frontend for chatbots that provides a modern and intuitive user interface. It supports various backends, including the OpenAI API, which we will use to connect to our DeepInfra model. OpenWebUI is highly customizable and can be easily integrated into your existing web applications.

Installing OpenWebUI Using Docker

Install Docker

If you don't have Docker installed yet, you can download and install it from the official Docker website.

Run the OpenWebUI Docker Image

The reason we’re using Docker to create our local OpenWebUI is because it’s so easy to set up. To run OpenWebUI using Docker, execute the following command in your terminal. This command will start the OpenWebUI container, map port 3000 on your host to port 8080 in the container, and set the necessary environment variables to connect to your DeepInfra backend.

docker run -d -p 3000:8080 \

-v open-webui:/app/backend/data \

-e OPENAI_API_BASE_URLS="https://api.deepinfra.com/v1/openai" \

-e OPENAI_API_KEYS="{Enter Your DeepInfra API Key}" \

--name open-webui \

--restart always \

ghcr.io/open-webui/open-webui:main

Config Explanation

Environment Variables :

OPENAI_API_BASE_URLS: Set this to the DeepInfra API endpoint URL.

OPENAI_API_KEYS: Replace {Enter Your DeepInfra API Key} with your actual DeepInfra API key.

Persistent Data :

The -v open-webui:/app/backend/data option ensures that the data directory is persisted across container restarts.

Port Mapping :

-p 3000:8080 maps port 3000 on your host machine to port 8080 in the Docker container.

Container Name :

--name open-webui names the container for easy reference.

Restart Policy :

--restart always ensures that the container will automatically restart if it stops.

Open your web browser and navigate to http://localhost:3000 to see your assistant in action. It should look something like this

Oh and btw, I created this article using the Qwen model. Judge for yourself if it was any good. By following these steps, you will have a fully functional and personal assistant with a sophisticated frontend using OpenWebUI, connected to your DeepInfra backend. You can now start interacting with your assistant and integrating it into your projects.