Qwen AI is the latest AI model released by Alibaba Group. While Copilot, DeepSeek, ChatGPT, and other popular AI tools are available for Android, iOS, and Windows platforms, Qwen is currently available as a web app. You can access and use this tool by visiting its official website. However, this requires you to sign up on the website. Additionally, you should also have an active internet connection. But do you know you can run Qwen AI locally on your Windows 11/10 PC? This article shows you how to do that.
How to use Qwen AI on the web?
To use Qwen AI from Alibaba on the web, visit chat.qwenlm.ai and enter your query. Continue chatting as you would in any AI tool. You will have to create an account and log in to use Qwen.
How to run Qwen AI locally on Windows 11/10
To run Qwen locally on your Windows PC, you need to install the following two tools:
- Ollama
- Docker
Ollama is a free and open-source project that allows users to run LLMs (large language models) on their local machines.
Docker is an open-source platform for developing, shipping, and running applications. Developers can share their work by creating containers in Docker.
In this guide, we will explain how you run Qwen AI locally on your Windows PC by creating a container in the Docker app.
How to install Ollama on Windows
The first step is to install Ollama on your system. Follow the steps provided below:
- Visit the Ollama’s official website.
- Click on the Download button, select the Windows platform, and proceed with downloading Ollama for your Windows machine.
- When the download completes, run the installer file to install Olama on your system.
- Now, launch Ollama and let it run in the background.
Install Qwen AI model locally
Now, the next step is to install the Qwen AI model on your system locally. Go to the official website of Ollama and click on the Models tab. Here, you will see all the AI models. Search for the Qwen. Then it will show you all the available Qwen AI models, including Qwen, Qwen2, Qwen2.5, etc. Click on the model that you want to deploy on your PC.
In this guide, we will see how to install the Qwen2.5 model locally on a Windows machine. Each Qwen model has different parameters. The Qwen2.5 model has parameters 0.5b, 1.5b, 3b, 7b, etc. Qwen 0.5b is the smallest and fastest parameter. It consists of 0.5 billion parameters and is good for basic tasks. The 1.5b parameter of Qwen AI is more advanced than the Qwen 0.5b parameter. Similarly, other large parameters are more advanced parameters than the previous ones.
Installing the large parameters requires a high-end PC. Therefore, install the one that your computer can run smoothly. Also, larger parameters require more disk space. Select the parameter from the drop-down and copy its respective command. For example, the command to install Qwen2.5 AI with 0.5b parameter is:
ollama run qwen2.5:0.5b
Open the Command Prompt, paste the copied command, and hit Enter. Do not close the Command Prompt before the completion of the command you gave.
When the command executes successfully, Qwen AI will be installed on your computer. Now, you can chat with the Qwen AI in CMD. If you want to chat with the Qwen AI the next time, open the Command Prompt and run the same command. To install another model of Qwen AI, copy the required command from the Ollama website and run it in the CMD.
Installing Docker on Windows
You can use Qwen AI in CMD on your Windows machine. However, it does not save your chat history. Moreover, installing a dedicated UI for Qwen AI also makes it easier to use. For this, you have to install Docker.
Visit the official website of Docker and download its Windows-supported version. Now, install the app on your PC. You need to create an account on Docker’s website to download and install it on your system.
After the Docker installation is completed, run Docker and make sure it keeps running in the background. Now, open the Command Prompt and execute the following command:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
The above command will create a container in the Docker app. It will take some time to complete the above-command. When the command completes, open the Docker app. You will see a container in the Containers section.
Run Qwen AI locally on Windows
Now, the last step is to run the Qwen AI locally on your system in the web browser. Open the Docker app and click on the 3000:8080 link in the Containers section. This will open the localhost with port 3000 in your default web browser. Now, create an account. Remember the login details, as the same login details will be required by you to use the Qwen AI every time.
Do note that to run Qwen AI locally in your web browser, both Ollama and the respective Docker Container should be running in the background.
That’s it.
Can I use AI on my PC?
Yes, you can use AI tools on your PC. If your system does not have supported hardware, you can use the cloud AI tools. Such tools are hosted on their servers. Hence, they do not put a load on your system. However, if you want to deploy an AI tool locally on your system, your system should have supported hardware.
Is Qwen LLM open-source?
Some models of the Qwen AI are open source, including Qwen2.5-7B-Instruct-1M and Qwen2.5-14B-Instruct-1M.