admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,026 Commits 137 Branches 188 Tags
Commit Graph

8886 Commits

Author SHA1 Message Date
chyezh
b3ee1a4bd5
fix: lost DroppedAt field when DropCollection (#33766) 
issue: #33097

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-12 20:21:56 +08:00
yihao.dai
9a3e4080f1
enhance: Add comment for channel cp updater (#33759) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-12 20:01:55 +08:00
Cai Yudong
9d4535ce0b
enhance: Handle Float16Vector/BFloat16Vector numpy bulk insert as same as BinaryVector (#33760) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-06-12 17:17:55 +08:00
Buqian Zheng
47b04ea167
enhance: support sparse cardinal hnsw index (#33656) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-12 16:57:55 +08:00
Buqian Zheng
8cb350598c
enhance: Improve GetVectorById of Sparse Float Vector (#33209) 
issue: #29419

* sparse float vector to support raw data mmap

For get vector from chunk cache, I added a unit test but marking it as
skipped due to a known issue. I have tested it locally.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-12 10:09:55 +08:00
coldWater
6b9901c59f
enhance: add a semaphore for CompactionExecutor (#33558) 
#33182

---------

Signed-off-by: coldWater <254244460@qq.com>
 2024-06-11 17:25:55 +08:00
Chun Han
f7af323d1e
fix: sync partitiion stats blocking balance task(#33741) (#33742) 
related: #33741

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-11 14:21:56 +08:00
Jiquan Long
ecf2bcee42
enhance: speed up array-equal operator via inverted index (#33633) 
fix: #33632

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-06-11 14:13:54 +08:00
chyezh
2b7ee1968f
enhance: new messsage interface for log service (#33286) 
issue: #33285

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-11 10:38:01 +08:00
chyezh
8ca5ced821
fix: async warmup will be blocked by state lock (#33686) 
issue: #33685

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-10 21:59:53 +08:00
yihao.dai
b1d46eb34b
fix: Fix multiple vector fields import (#33723) 
1. Fix dim mismatch with multi-vector fields and JSON import
2. Enhance: do not display file ID in GetImportResponse.

issue: https://github.com/milvus-io/milvus/issues/33681,
https://github.com/milvus-io/milvus/issues/33682

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:57:54 +08:00
yihao.dai
eb5d4de390
fix: Check if the import job exists (#33672) 
issue: https://github.com/milvus-io/milvus/issues/33671

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:51:55 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-10 21:34:08 +08:00
chyezh
f53ab54c5d
enhance: async cgo utility (#33133) 
issue: #30926, #33132

- implement future-based cgo utility.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-09 22:55:53 +08:00
yihao.dai
3540eee977
enhance: Support L0 import (#33514) 
issue: https://github.com/milvus-io/milvus/issues/33157

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-07 14:17:20 +08:00
zhagnlu
6ce9df913f
fix: clean vector memory (#33692) 
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-07 10:37:54 +08:00
wei liu
0a3d456688
enhance: add restful api to trigger component stop (#32076) 
issue: #32698
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-07 10:35:54 +08:00
smellthemoon
c61fb1eff5
enhance: do check when add not empty logpath (#33640) 
meta only store logid

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-06-07 10:19:51 +08:00
SimFG
ecee7d90d4
enhance: try to speed up the loading of small collections (#33570) 
- issue: #33569

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-07 08:25:53 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data (#33651) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
 2024-06-06 17:37:51 +08:00
cai.zhang
cfea3f43cf
fix: Don't sync L0 segments to channel watcher (#33664) 
issue: #33540

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-06 15:59:50 +08:00
XuanYang-cn
4dd0c54ca0
fix: Fix l0 compactor may cause DN from OOM (#33554) 
See also: #33547

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-06 14:33:52 +08:00
congqixia
f6e251514f
fix: Write back dbid modification for nonDB id collection (#33641) 
See also #33608

Make `fixDefaultDBIDConsistency` also write back collection dbid
modification when nonDB id collection is found.

This fix shall prevent dropped collections of this kind show up again
after dropping and restart.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-06 14:29:53 +08:00
wei liu
b69740c8f3
enhance: Remove unnecessary log info during load segment (#33663) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-06 14:13:50 +08:00
XuanYang-cn
68c9e7db8c
fix: Sync dropped segment for dropped partition (#33331) 
See also: #33330

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-06 10:25:52 +08:00
sre-ci-robot
fd191dd7db
[automated] Update Knowhere Commit (#33655) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-06-06 01:53:50 +08:00
cai.zhang
feeb869ff9
enhance: Remove compaction plans on the datanode (#33548) 
issue: #33546

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-05 20:27:51 +08:00
cqy123456
703fc73f71
enhance: disk index support binary vector (#33631) 
issue:https://github.com/milvus-io/milvus/issues/22837
related https://github.com/milvus-io/milvus/pull/33575

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-05 19:37:57 +08:00
cai.zhang
412ccfbb20
enhance: Refine IndexNode code and ensure compatibility (#33458) 
issue: #33432 , #33183

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-05 19:35:52 +08:00
zhagnlu
8ad26093ba
fix: fix load failure (#33599) 
issue: #33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-05 19:19:51 +08:00
aoiasd
b9bafc76b4
enhance: access log support print with consistency level (#33503) 
relate: https://github.com/milvus-io/milvus/issues/33563

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-06-05 17:13:50 +08:00
yihao.dai
01ce32caa1
enhance: Print more disk quota info (#33596) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 16:51:56 +08:00
yihao.dai
bbdf99a45e
fix: Fix import segment size is uneven (#33605) 
The data coordinator computed the appropriate number of import segments,
thus when importing in the data node, one can randomly select a segment.

issue: https://github.com/milvus-io/milvus/issues/33604

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 15:41:51 +08:00
Gao
545d4725fb
fix: correct get vector data size for bf16/fp16/binary vector (#33377) 
related #22837

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-05 14:31:57 +08:00
congqixia
597f4c5e03
enhance: Make hasMoreResult accurate when hit number larger than limit (#33609) 
See also milvus-io/milvus-sdk-go#756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-05 11:51:51 +08:00
yihao.dai
35532a3e7d
fix: Fill stats log id and check validity (#33477) 
1. Fill log ID of stats log from import
2. Add a check to validate the log ID before writing to meta

issue: https://github.com/milvus-io/milvus/issues/33476

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 11:17:56 +08:00
congqixia
25080bb455
enhance: Print UseDefaultConsistency param in read requests (#33617) 
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-05 10:27:50 +08:00
zhenshan.cao
ac4f3997ce
enhance: Reconstructing Compaction to possess persistence capability (#33265) 
issue #33586

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-06-05 10:17:50 +08:00
aoiasd
387b7cd7f4
enhance:avoid maintain checkpoint info in sync manager (#33413) 
relate: https://github.com/milvus-io/milvus/issues/32915

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-06-05 10:05:50 +08:00
jaime
8858fcb40a
fix: fix loaded entity num is inaccurate (#33521) 
issue: #33520

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-04 20:09:54 +08:00
zhagnlu
c6f8a73bb2
enhance: optimize some cache to reduce memory usage (#33534) 
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-04 14:09:47 +08:00
congqixia
7f4698f4a7
enhance: Use map PK to timestamp in buffer insert (#33566) 
Related to #27675

Store pk to minimal timestamp in `inData` instead of bloom filter to
check whether some delete entry hit current insert batch

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-04 10:07:48 +08:00
sre-ci-robot
d25c755480
[automated] Update Knowhere Commit (#33573) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-06-04 01:55:46 +08:00
SimFG
44d7e03e56
fix: reset the RootCoordQuotaStates metric before recording this metric (#33553) 
- issue: #33539

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-03 21:09:47 +08:00
congqixia
2b285e5573
fix: Wrap init segcore tracing with golang timeout (#33494) 
See also #33483

Wrap `C.InitTrace` & `C.SetTrace` with timeout preventing otlp
initializtion hangs forever when endpoint is not set correctly

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-03 19:25:51 +08:00
wei liu
34c6a989ab
enhance: Avoid load bf in delegator when qn worker has no more memory (#33557) 
query coord send load request to delegator, delegator load bf first,
then forward load request to qn worker. but when qn worker has no more
memory, it will return load failed immediatelly. then delegator roll
back the loaded bf. query coord wil retry the load request, and
delegator will load and roll back bf again and again.

this PR delay the loading bf step until load segment succeed in worker.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-03 19:23:45 +08:00
yiwangdr
180d754158
fix: speed up segment lookup via channel name in datacoord (#33530) 
issue: #33342

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-06-03 14:47:47 +08:00
yihao.dai
7a2127b09f
enhance: Avoid redundant meta operations of import (#33518) 
issue: https://github.com/milvus-io/milvus/issues/33513

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-03 14:15:53 +08:00
XuanYang-cn
0382628668
enhance: Add more tracing for l0 compactor (#33435) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-03 10:19:49 +08:00
smellthemoon
2c7bb0b8ac
fix: replace removeWithPrefix with remove to avoid delete redundantly (#33328) 
#33288

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-31 18:05:45 +08:00