admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,515 Commits 137 Branches 188 Tags
Commit Graph

1155 Commits

Author SHA1 Message Date
chyezh
b3ee1a4bd5
fix: lost DroppedAt field when DropCollection (#33766) 
issue: #33097

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-12 20:21:56 +08:00
Chun Han
f7af323d1e
fix: sync partitiion stats blocking balance task(#33741) (#33742) 
related: #33741

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-11 14:21:56 +08:00
yihao.dai
b1d46eb34b
fix: Fix multiple vector fields import (#33723) 
1. Fix dim mismatch with multi-vector fields and JSON import
2. Enhance: do not display file ID in GetImportResponse.

issue: https://github.com/milvus-io/milvus/issues/33681,
https://github.com/milvus-io/milvus/issues/33682

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:57:54 +08:00
yihao.dai
eb5d4de390
fix: Check if the import job exists (#33672) 
issue: https://github.com/milvus-io/milvus/issues/33671

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:51:55 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-10 21:34:08 +08:00
yihao.dai
3540eee977
enhance: Support L0 import (#33514) 
issue: https://github.com/milvus-io/milvus/issues/33157

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-07 14:17:20 +08:00
smellthemoon
c61fb1eff5
enhance: do check when add not empty logpath (#33640) 
meta only store logid

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-06-07 10:19:51 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data (#33651) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
 2024-06-06 17:37:51 +08:00
cai.zhang
cfea3f43cf
fix: Don't sync L0 segments to channel watcher (#33664) 
issue: #33540

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-06 15:59:50 +08:00
cai.zhang
feeb869ff9
enhance: Remove compaction plans on the datanode (#33548) 
issue: #33546

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-06-05 20:27:51 +08:00
yihao.dai
01ce32caa1
enhance: Print more disk quota info (#33596) 
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 16:51:56 +08:00
yihao.dai
35532a3e7d
fix: Fill stats log id and check validity (#33477) 
1. Fill log ID of stats log from import
2. Add a check to validate the log ID before writing to meta

issue: https://github.com/milvus-io/milvus/issues/33476

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-05 11:17:56 +08:00
zhenshan.cao
ac4f3997ce
enhance: Reconstructing Compaction to possess persistence capability (#33265) 
issue #33586

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-06-05 10:17:50 +08:00
yiwangdr
180d754158
fix: speed up segment lookup via channel name in datacoord (#33530) 
issue: #33342

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-06-03 14:47:47 +08:00
yihao.dai
7a2127b09f
enhance: Avoid redundant meta operations of import (#33518) 
issue: https://github.com/milvus-io/milvus/issues/33513

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-03 14:15:53 +08:00
cai.zhang
77637180fa
enhance: Periodically synchronize segments to datanode watcher (#33420) 
issue: #32809

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-05-30 13:37:44 +08:00
congqixia
54797b4286
enhance: Refine frequent log in datacoord (#33449) 
This PR changes:
- Frequent `ListIndexes` success log to debug level
- Aggregate collection missing log after collection dropped in
`meta.GetCollectionIndexFilesSize`

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-29 19:15:43 +08:00
jaime
3d29907b6e
enhance: decrease cpu overhead during filter segments on datacoord (#33130) 
issue: #33129

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-05-28 19:17:43 +08:00
XuanYang-cn
5e39aa9272
enhance: Make channel meta able to writer 200k plus segments (#33279) 
See also: #33125

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-28 12:33:42 +08:00
SimFG
2453181218
fix: not found database name in the datacoord meta object (#33411) 
- issue: #33410

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-28 10:09:48 +08:00
yiwangdr
e895cfed84
fix: reduce redundant map operations in datacoord (#33343) 
More refactories will be added.
issue: #33342

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-24 12:47:40 +08:00
yihao.dai
7730b910b9
enhance: Decouple compaction from shard (#33138) 
Decouple compaction from shard, remove dependencies on shards (e.g.
SyncSegments, injection).

issue: https://github.com/milvus-io/milvus/issues/32809

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-24 09:07:41 +08:00
cai.zhang
be77ceba84
enhance: Use proto for passing info in cgo (#33184) 
issue: #33183

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-05-23 10:31:40 +08:00
XuanYang-cn
22bddde5ff
enhance: Tidy compactor and remove dup codes (#32198) 
See also: #32451

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-23 09:53:40 +08:00
XuanYang-cn
819a624753
fix: Return error when startup Delete/AddNode fail (#33193) 
See also: #33151, #33149

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-22 11:17:38 +08:00
cai.zhang
ed39a38953
enhance: Reduce the frequency of logs describing indexing failures (#33212) 
issue: #33001 #33102

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-05-21 19:27:39 +08:00
yihao.dai
017fd7bc25
enhance: Select L2 segments in L0Compaction as well (#32991) 
/kind improvement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-21 16:13:39 +08:00
congqixia
f31a20faad
fix: [Backport] Mark channel checkpoint dropped prevent cp lag metrics leakage (#32454) (#33198) 
Cherry-pick from 2.3
pr: #32454
See also #31506 #31508

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-21 11:59:39 +08:00
jaime
0d99db23b8
fix: metrics leak on the coord nodes (#33075) 
issue: #32980

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-05-20 22:03:39 +08:00
congqixia
f76d16780b
enhance: Refine channel mgr v2 implementation (#33156) 
Related to #25309

- Remove ctx from struct
- Add ctx parameters for internal check logic methods
- Add Waitgroup to make sure worker goroutine quit before close returns

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-20 14:13:38 +08:00
yihao.dai
32560263fa
enhance: Query slot for compaction task (#32881) 
Query slot of compaction in datanode, and transfer the control logic for
limiting compaction tasks from datacoord to the datanode.

issue: https://github.com/milvus-io/milvus/issues/32809

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-17 18:19:38 +08:00
congqixia
8cf2cf5c94
enhance: Add go-deadlock as unittest only dependency (#33063) 
See also #33062

This PR:

- Add `lock.RWMutex` & `lock.Mutex` alias to switch implementation based
  on build flags
- When build flags has `test` in it, use `go-deadlock` to detect
  possible deadlocks
- Replace all `sync.RWMutex` & `sync.Mutex` in datacoord pkg

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-15 16:33:34 +08:00
cai.zhang
4382cf5283
enhance: Reduce the frequency of index ralted logs (#33050) 
issue: #33001

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-05-15 14:37:34 +08:00
wayblink
259bc97a2b
fix: Fix segments lost in flush response (#33061) 
#33055

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-05-15 13:49:34 +08:00
cai.zhang
6ea7633bd5
enhance: Add memory size for binlog (#33025) 
issue: #33005
1. add `MemorySize` field for insert binlog.
2. `LogSize` means the file size in the storage object.
3. `MemorySize` means the size of the data in the memory.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2024-05-15 12:59:34 +08:00
jaime
f48a7ff8ff
enhance: use Delete instead of DeletePartialMatch to remove metrics (#33029) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-05-14 18:49:33 +08:00
congqixia
dea7faf394
fix: Remove RLock re-entry in GetNumRowsOfPartition (#33045) 
Resolves #33044

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-14 17:51:32 +08:00
XuanYang-cn
40e5c0be70
fix: Enable to balance released standby channel (#32986) 
See also: #32879

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-14 11:51:32 +08:00
jaime
ba625835bc
enhance: Add metrics for segment index files size (#32979) 
issue:#32980

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-05-13 17:59:32 +08:00
XuanYang-cn
bc015df962
fix: [skip-e2e] Fix unstable policy_test (#32883) 
See also: #32849

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-09 15:39:40 +08:00
congqixia
cedb33ceec
enhance: Improve datacoord segment filtering with collection (#32831) 
See also #32165

This PR modify the `SelectSegments` interface to utilizing collection id
information when selecting segment with provided collection

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-08 21:37:29 +08:00
XuanYang-cn
0b0d279515
fix: [skip-e2e] Fix AvgAssignByCount unstable (#32850) 
See also: #32849

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-08 19:09:35 +08:00
yihao.dai
a8db16ad4b
enhance: Change ddl log to info (#32838) 
/kind improvement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-08 14:17:50 +08:00
XuanYang-cn
6843d6d376
fix: Compaction trigger choose 2 same segments (#32800) 
DataNode would stuck at compactor try to lock the
same segmentID

See also: #32765

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-07 19:01:31 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036) 
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-07 15:49:30 +08:00
Bingyi Sun
fecd9c21ba
feat: LRU cache implementation (#32567) 
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2024-05-06 20:29:30 +08:00
congqixia
6af5275066
fix: Refine index param check for Create/Alter Index (#32712) 
See also #32711

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-06 15:43:29 +08:00
chyezh
77f4f0c5a8
fix: gc only do once (#32722) 
issue: #29655

produced by pr: #31740

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-30 14:19:27 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse (#32044) 
#32609

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-04-30 09:45:27 +08:00
jaime
3b5cbed86a
enhance: add db label for deletion and bulkinsert operation (#32611) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-26 15:53:29 +08:00