


Docker completes local deployment of LLama3 open source large model in three minutes
Apr 26, 2024 am 10:19 AMOverview
LLaMA-3 (Large Language Model Meta AI 3) is a large-scale open source generative artificial intelligence model developed by Meta Company. It has no major changes in model structure compared with the previous generation LLaMA-2.
The LLaMA-3 model is divided into different scale versions, including small, medium and large, to adapt to different application requirements and computing resources. The parameter size of small models is 8B, the parameter size of medium models is 70B, and the parameter size of large models reaches 400B. However, during training, the goal is to achieve multi-modal and multi-language functionality, and the results are expected to be comparable to GPT 4/GPT 4V.
Installing Ollama
Ollama is an open source large language model (LLM) service tool that allows users to run and deploy large language models on their local machine. Ollama is designed as a framework that simplifies the process of deploying and managing large language models in Docker containers, making the process quick and easy. Users can quickly run open source large-scale language models such as Llama 3 locally through simple command line operations.
Official website address: https://ollama.com/download
Picture
Ollama is a tool that supports multiple platforms. Includes Mac and Linux, and provides Docker images to simplify the installation process. Users can import and customize more models by writing a Modelfile, which is similar to the role of a Dockerfile. Ollama also features a REST API for running and managing models, and a command-line toolset for model interaction.
Ollama service startup log
Picture
Model management
Download model
ollama pull llama3:8b
The default download is llama3:8b. The colon before the colon here represents the model name, and the colon after the tag represents the tag. You can view all tags of llama3 from here
Pictures
Model Test
Note: If you want the model to reply in Chinese, please enter: Hello! Please reply in Chinese
Picture
Configure Open-WebUI
Run under CPU
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
Picture
Access
Enter the address http://127.0.0.1:3000 to access
Picture
The first visit requires registration. Here I register an account. After registration is completed, the login is successful
Picture
Switch Chinese language
Picture
Download llama3:8b model
llama3:8b
Picture
Download completed
Picture
Use
Select model
Picture
Use model
Picture
Note: If you want the model to reply in Chinese, please enter: Hello! Please reply in Chinese
Picture
Memory
## picture
The above is the detailed content of Docker completes local deployment of LLama3 open source large model in three minutes. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undress AI Tool
Undress images for free

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

As a pioneer in the digital world, Bitcoin’s unique code name and underlying technology have always been the focus of people’s attention. Its standard code is BTC, also known as XBT on certain platforms that meet international standards. From a technical point of view, Bitcoin is not a single code style, but a huge and sophisticated open source software project. Its core code is mainly written in C and incorporates cryptography, distributed systems and economics principles, so that anyone can view, review and contribute its code.

Linuxcanrunonmodesthardwarewithspecificminimumrequirements.A1GHzprocessor(x86orx86_64)isneeded,withadual-coreCPUrecommended.RAMshouldbeatleast512MBforcommand-lineuseor2GBfordesktopenvironments.Diskspacerequiresaminimumof5–10GB,though25GBisbetterforad

There are three main ways to set environment variables in PHP: 1. Global configuration through php.ini; 2. Passed through a web server (such as SetEnv of Apache or fastcgi_param of Nginx); 3. Use putenv() function in PHP scripts. Among them, php.ini is suitable for global and infrequently changing configurations, web server configuration is suitable for scenarios that need to be isolated, and putenv() is suitable for temporary variables. Persistence policies include configuration files (such as php.ini or web server configuration), .env files are loaded with dotenv library, and dynamic injection of variables in CI/CD processes. Security management sensitive information should be avoided hard-coded, and it is recommended to use.en

To enable PHP containers to support automatic construction, the core lies in configuring the continuous integration (CI) process. 1. Use Dockerfile to define the PHP environment, including basic image, extension installation, dependency management and permission settings; 2. Configure CI/CD tools such as GitLabCI, and define the build, test and deployment stages through the .gitlab-ci.yml file to achieve automatic construction, testing and deployment; 3. Integrate test frameworks such as PHPUnit to ensure that tests are automatically run after code changes; 4. Use automated deployment strategies such as Kubernetes to define deployment configuration through the deployment.yaml file; 5. Optimize Dockerfile and adopt multi-stage construction

Using the correct PHP basic image and configuring a secure, performance-optimized Docker environment is the key to achieving production ready. 1. Select php:8.3-fpm-alpine as the basic image to reduce the attack surface and improve performance; 2. Disable dangerous functions through custom php.ini, turn off error display, and enable Opcache and JIT to enhance security and performance; 3. Use Nginx as the reverse proxy to restrict access to sensitive files and correctly forward PHP requests to PHP-FPM; 4. Use multi-stage optimization images to remove development dependencies, and set up non-root users to run containers; 5. Optional Supervisord to manage multiple processes such as cron; 6. Verify that no sensitive information leakage before deployment

Building an independent PHP task container environment can be implemented through Docker. The specific steps are as follows: 1. Install Docker and DockerCompose as the basis; 2. Create an independent directory to store Dockerfile and crontab files; 3. Write Dockerfile to define the PHPCLI environment and install cron and necessary extensions; 4. Write a crontab file to define timing tasks; 5. Write a docker-compose.yml mount script directory and configure environment variables; 6. Start the container and verify the log. Compared with performing timing tasks in web containers, independent containers have the advantages of resource isolation, pure environment, strong stability, and easy expansion. To ensure logging and error capture

There are four ways to obtain command help in Linux: First, use --help to view basic usage, which is suitable for quickly understanding common options and parameters of commands; second, use man to view the complete man page, providing detailed command descriptions and examples; third, use info to view structured help, which is suitable for information navigation of complex commands such as gcc and make; fourth, refer to network resources and communities, such as Linux China, StackOverflow and other platforms to obtain Chinese materials or solve specific problems. It is recommended for beginners to master it step by step from --help and man.

Add useradd or adduser commands commonly used by users in Linux. 1. When using useradd, you need to manually set the password and home directory. Add the -m parameter to create the home directory; 2. You can specify the shell, group and UID through parameters such as -s, -G, and -u; 3. Adduser is an interactive command, suitable for novices to automatically complete the configuration; 4. Pay attention to permissions, username uniqueness and home directory permissions; 5. Userdel can be used to delete users and home directory by mistake. Mastering these key points allows you to manage users efficiently and securely.
