外观
Milvus 单机部署指南
本指南将介绍如何在单机环境中快速部署 Milvus 向量数据库,适合开发测试、学习和小规模应用场景。
概述
单机部署特点
- 快速部署:几分钟内完成安装
- 资源占用少:适合开发测试环境
- 易于维护:配置简单,易于排查问题
- 功能完整:支持所有核心功能
适用场景
- 开发和测试环境
- 学习和实验
- 小规模应用(百万级向量)
- 原型验证
环境要求
硬件要求
最低配置
- CPU: 4核心
- 内存: 8GB
- 存储: 50GB SSD
- 网络: 100Mbps
推荐配置
- CPU: 8核心
- 内存: 16GB
- 存储: 200GB SSD
- 网络: 1Gbps
软件要求
- 操作系统: Linux (Ubuntu 18.04+) / macOS / Windows
- Docker: 19.03+
- Docker Compose: 1.25+(可选)
部署方式
方式一:Docker 部署(推荐)
1. 拉取镜像
bash
docker pull milvusdb/milvus:v2.3.32. 启动 Milvus
bash
docker run -d \
--name milvus-standalone \
-p 19530:19530 \
-p 9091:9091 \
-v $(pwd)/milvus:/var/lib/milvus \
milvusdb/milvus:v2.3.3 \
milvus run standalone3. 验证运行状态
bash
docker ps | grep milvus
docker logs milvus-standalone方式二:Docker Compose 部署
1. 创建配置文件
创建 docker-compose.yml:
yaml
version: '3.5'
services:
etcd:
container_name: milvus-etcd
image: quay.io/coreos/etcd:v3.5.5
environment:
- ETCD_AUTO_COMPACTION_MODE=revision
- ETCD_AUTO_COMPACTION_RETENTION=1000
- ETCD_QUOTA_BACKEND_BYTES=4294967296
- ETCD_SNAPSHOT_COUNT=50000
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
healthcheck:
test: ["CMD", "etcdctl", "endpoint", "health"]
interval: 30s
timeout: 20s
retries: 3
minio:
container_name: milvus-minio
image: minio/minio:RELEASE.2023-03-20T20-16-18Z
environment:
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
ports:
- "9001:9001"
- "9000:9000"
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
command: minio server /minio_data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
standalone:
container_name: milvus-standalone
image: milvusdb/milvus:v2.3.3
command: ["milvus", "run", "standalone"]
security_opt:
- seccomp:unconfined
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
interval: 30s
start_period: 90s
timeout: 20s
retries: 3
ports:
- "19530:19530"
- "9091:9091"
depends_on:
- "etcd"
- "minio"
networks:
default:
name: milvus2. 启动服务
bash
docker-compose up -d3. 查看状态
bash
docker-compose ps
docker-compose logs -f standalone4. 停止服务
bash
docker-compose down方式三:二进制部署
1. 下载安装包
bash
wget https://github.com/milvus-io/milvus/releases/download/v2.3.3/milvus-standalone-linux.tar.gz
tar -xzf milvus-standalone-linux.tar.gz2. 安装依赖
bash
sudo apt-get update
sudo apt-get install -y etcd-server minio3. 启动 etcd
bash
etcd --listen-client-urls=http://0.0.0.0:2379 \
--advertise-client-urls=http://localhost:2379 &4. 启动 MinIO
bash
minio server /data --address ":9000" &5. 启动 Milvus
bash
export ETCD_ENDPOINTS=localhost:2379
export MINIO_ADDRESS=localhost:9000
./milvus run standalone基础使用
安装 Python SDK
bash
pip install pymilvus连接 Milvus
python
from pymilvus import connections, utility
connections.connect(
alias="default",
host='localhost',
port='19530'
)
print(f"Milvus version: {utility.get_server_version()}")创建集合
python
from pymilvus import Collection, FieldSchema, CollectionSchema, DataType
fields = [
FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=False),
FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128)
]
schema = CollectionSchema(fields=fields, description="示例集合")
collection = Collection(
name="example_collection",
schema=schema
)
print(f"集合 {collection.name} 创建成功")插入数据
python
import numpy as np
num_entities = 10000
entities = [
list(range(num_entities)),
[[np.random.random() for _ in range(128)] for _ in range(num_entities)]
]
insert_result = collection.insert(entities)
print(f"插入了 {collection.num_entities} 条数据")创建索引
python
index_params = {
"metric_type": "L2",
"index_type": "IVF_FLAT",
"params": {"nlist": 128}
}
collection.create_index(
field_name="embedding",
index_params=index_params
)
print("索引创建成功")加载集合
python
collection.load()向量检索
python
search_params = {
"metric_type": "L2",
"params": {"nprobe": 10}
}
search_vector = [[np.random.random() for _ in range(128)]]
results = collection.search(
data=search_vector,
anns_field="embedding",
param=search_params,
limit=10,
expr=None
)
for hits in results:
for hit in hits:
print(f"ID: {hit.id}, Distance: {hit.distance}")配置优化
内存配置
创建配置文件 milvus.yaml:
yaml
standalone:
memory:
limit: 8GB
queryNode:
segcore:
chunkRows: 1024
cache:
enabled: true
memoryLimit: 4GB
dataNode:
flush:
insertBufferSize: 16MB
insertBufferPoolSize: 1GB启动时指定配置文件:
bash
docker run -d \
--name milvus-standalone \
-p 19530:19530 \
-v $(pwd)/milvus.yaml:/milvus/configs/milvus.yaml \
-v $(pwd)/milvus:/var/lib/milvus \
milvusdb/milvus:v2.3.3 \
milvus run standalone存储配置
使用外部存储
yaml
minio:
address: your-minio-server.com
port: 9000
accessKeyID: your-access-key
secretAccessKey: your-secret-key
useSSL: true
bucketName: milvus-data数据压缩
yaml
dataCoord:
enableCompaction: true
compaction:
enableAutoCompaction: true性能调优
索引优化
python
index_params = {
"metric_type": "L2",
"index_type": "HNSW",
"params": {
"M": 16,
"efConstruction": 256
}
}
collection.create_index(
field_name="embedding",
index_params=index_params
)查询优化
python
search_params = {
"metric_type": "L2",
"params": {
"ef": 64
}
}
results = collection.search(
data=search_vectors,
anns_field="embedding",
param=search_params,
limit=100
)数据管理
数据导入
从文件导入
python
import pandas as pd
df = pd.read_csv('vectors.csv')
entities = [
df['id'].tolist(),
df['embedding'].apply(eval).tolist()
]
collection.insert(entities)批量导入
python
import numpy as np
batch_size = 10000
total_records = 100000
for i in range(0, total_records, batch_size):
batch = [
list(range(i, i + batch_size)),
np.random.random((batch_size, 128)).tolist()
]
collection.insert(batch)
print(f"已插入 {i + batch_size} 条记录")数据导出
python
results = collection.query(
expr="id >= 0",
output_fields=["id", "embedding"]
)
import json
with open('export.json', 'w') as f:
json.dump(results, f)数据备份
使用 Milvus Backup
bash
docker run --rm \
-v $(pwd)/backup:/backup \
-e MILVUS_ADDRESS=localhost:19530 \
milvusdb/milvus-backup:latest \
milvus-backup create -n backup-$(date +%Y%m%d)监控与日志
健康检查
bash
curl http://localhost:9091/healthz性能指标
访问 Prometheus 指标:
bash
curl http://localhost:9091/metrics日志查看
bash
docker logs milvus-standalone -f配置日志级别
yaml
log:
level: debug
file:
rootPath: /var/lib/milvus/logs
maxSize: 300
maxAge: 10
maxBackups: 20升级迁移
版本升级
1. 备份数据
bash
docker exec milvus-standalone \
milvus-backup create -n backup-before-upgrade2. 停止服务
bash
docker stop milvus-standalone
docker rm milvus-standalone3. 拉取新版本
bash
docker pull milvusdb/milvus:v2.3.44. 启动新版本
bash
docker run -d \
--name milvus-standalone \
-p 19530:19530 \
-v $(pwd)/milvus:/var/lib/milvus \
milvusdb/milvus:v2.3.4 \
milvus run standalone集群迁移
1. 导出数据
python
from pymilvus import connections, Collection
connections.connect(alias="source", host='source-host', port='19530')
collection = Collection("example_collection")
results = collection.query(
expr="",
output_fields=["*"],
limit=10000
)2. 导入到新环境
python
connections.connect(alias="target", host='target-host', port='19530')
target_collection = Collection("example_collection")
target_collection.insert(results)常见问题
1. 容器启动失败
问题: Docker 容器无法启动
解决方案:
bash
docker logs milvus-standalone
docker run --rm milvusdb/milvus:v2.3.3 milvus run standalone --help2. 连接超时
问题: 无法连接到 Milvus
解决方案:
bash
docker exec -it milvus-standalone bash
curl localhost:19530/v1/vector/collections
netstat -tlnp | grep 195303. 内存不足
问题: OOM (Out of Memory) 错误
解决方案:
yaml
standalone:
memory:
limit: 16GB4. 性能问题
问题: 查询速度慢
解决方案:
- 优化索引参数
- 增加内存配置
- 使用更快的存储
最佳实践
1. 开发环境
- 使用 Docker 部署
- 配置合适的资源限制
- 定期清理测试数据
2. 数据管理
- 合理设计集合结构
- 选择合适的索引类型
- 定期备份数据
3. 性能优化
- 使用批量操作
- 优化索引参数
- 监控系统资源
4. 安全建议
- 不要暴露到公网
- 配置访问控制
- 定期更新版本
参考资料
官方文档
相关教程
提示
单机部署适合开发测试,生产环境建议使用集群部署以获得更好的性能和可靠性。
信息
本指南基于 Milvus 2.3 版本编写,如需其他版本,请参考官方文档。
