admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-29 15:05:31 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,539 Commits 136 Branches 185 Tags
Commit Graph

851 Commits

Author SHA1 Message Date
congqixia
2fbc628994
feat: Support field partial load collection (#35416) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-20 16:49:02 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:16:55 +08:00
Zhen Ye
4d69898cb2
enhance: support single pchannel level transaction (#35289) 
issue: #33285

- support transaction on single wal.
- last confirmed message id can still be used when enable transaction.
- add fence operation for segment allocation interceptor.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-19 21:22:56 +08:00
wei liu
853d2f33e0
enhance: Trim space during parsing string list from string (#35546) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 20:48:55 +08:00
congqixia
2736a8b88c
enhance: Update Makefile to generate mockery (#35517) 
Some mockery cmd is out-of-date and fail to work. This PR update these
commands to match current pkg.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 18:44:53 +08:00
wei liu
ead37326f0
enhance: Add backup/restore privilege to admin group (#35515) 
issue: #35443

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 18:42:54 +08:00
wei liu
a570567644
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) 
issue: #35471

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 14:18:54 +08:00
Buqian Zheng
f4a91e135b
enhance: Allow empty sparse row (#34700) 
issue: #29419

* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-08-16 14:14:54 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 10:10:53 +08:00
SimFG
b2cc4b0776
feat: add the rbac msg and send them to the replicate channel (#35392) 
- issue: #35391

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-15 12:06:52 +08:00
Ted Xu
c6ae7d4d56
enhance: adding the msgchannel section in generated yaml (#35466) 
See #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-14 18:32:53 +08:00
congqixia
f99a761d87
enhance: Support dynamic field in SchemaHelper (#35461) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-14 18:04:59 +08:00
Reid Chan
6c135e5ea9
fix: key of DataCoordTicklePath is wrong (issue: #35259) (#35263) 
issue: [35259](https://github.com/milvus-io/milvus/issues/35259)

Signed-off-by: Reid Chan <reidchan@apache.org>
 2024-08-13 17:40:20 +08:00
chyezh
16b0aee97f
enhance: timetick interceptor optimization (#35287) 
issue: #33285

- remove redundant goroutine by using insepctor.
- remove the coutinous non-message timetick persistence
- periodically push the time tick forward without persistent timetick
message.
- add 'message type filter' deliver filter.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-12 18:58:25 +08:00
smellthemoon
c4a240acb1
enhance: memory value are displayed in MB (#35292) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-09 14:14:25 +08:00
wei liu
344dc6a9f8
enhance: enable to set load config in cluster level (#35169) 
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-07 12:38:21 +08:00
chyezh
c725416288
enhance: move streaming proto into pkg (#35284) 
issue: #33285

- move streaming related proto into pkg.
- add v2 message type and change flush message into v2 message.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-07 10:34:16 +08:00
Jiquan Long
b5022b0647
enhance: add metric to record disk quota (#35305) 
https://github.com/milvus-io/milvus/issues/35177

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-07 10:26:16 +08:00
Xiaofan
885d891106
Change minio use virtual config to not panic (#35327) 
minio use virtual should not panic if empty. User may use other object
storage instead

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-07 09:58:15 +08:00
presburger
a9352a073b
enhance: change gpu default mem pool size (#34159) 
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-08-06 10:24:15 +08:00
chyezh
14051fed7d
enhance: streaming service client (#34656) 
issue: #33285

- implement streaming service client.
- implement producing and consuming service client by streaming coord
client and streaming node client.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-05 21:38:15 +08:00
cai.zhang
6542c1ab0e
enhance: Add monitoring metrics for task execution time in datacoord (#35139) 
issue: #35138

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-05 16:26:17 +08:00
congqixia
a6903701a4
enhance: Remove stackerror imported unexpected (#35256) 
`stackerror` pkg is imported by accident and could be replaced by
cockroachdb errors lib. This PR removes import line and fix this
problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-05 14:48:16 +08:00
chyezh
9871966415
enhance: segment alloc interceptor (#34996) 
#33285

- add segment alloc interceptor for streamingnode.
- add add manual alloc segment rpc for datacoord.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-04 07:40:15 +08:00
yihao.dai
a4439cc911
enhance: Implement flusher in streamingNode (#34942) 
- Implement flusher to:
  - Manage the pipelines (creation, deletion, etc.)
  - Manage the segment write buffer
  - Manage sync operation (including receive flushMsg and execute flush)
- Add a new `GetChannelRecoveryInfo` RPC in DataCoord.
- Reorganize packages: `flushcommon` and `datanode`.

issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-02 18:30:23 +08:00
jaime
fcec4c21b9
fix: check collection health(queryable) fail for releasing collection (#34947) 
issue: #34946

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-02 17:20:15 +08:00
congqixia
db06b86594
enhance: Sync otlp dependency version to fix security issue (#35192) 
Related to #34434

otelgrpc DoS vulnerability due to unbound cardinality metrics

https://github.com/milvus-io/milvus/security/dependabot/91

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 16:22:20 +08:00
wei liu
f466129924
enhance: add segment state on data coord bin log size metrics (#35153) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 14:48:14 +08:00
aoiasd
3655ab10b2
fix: evict paramtable cache miss (#34771) 
relate: https://github.com/milvus-io/milvus/issues/33461

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-02 11:52:14 +08:00
congqixia
c64a078458
enhance: Support proxy/delegator qn client pooling (#35194) 
See also #35196

Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
wei liu
3b735b4b02
enhance: Refine param init for MmapDirPath (#35181) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 11:12:14 +08:00
Jiquan Long
4dbd023402
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel (#35185) 
fix: https://github.com/milvus-io/milvus/issues/35177

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-02 10:14:13 +08:00
cai.zhang
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size (#35076) 
issue: #34495

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 22:04:14 +08:00
yihao.dai
7721a28584
fix: Fix unstable TestDispatchToVchannels ut (#35163) 
issue: https://github.com/milvus-io/milvus/issues/35150

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-01 14:56:13 +08:00
Ted Xu
b4d0f4df0a
enhance: update param documents (#35152) 
See: #35161

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-01 12:38:13 +08:00
wei liu
e9d61daa3f
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 10:21:50 +08:00
congqixia
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib (#35140) 
See also #34394 #34252

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 10:01:49 +08:00
Gao
6695c6d0a3
enhance: add channel num for queryHook optimization (#35104) 
At most cases, data in each channel is almost evenly distributed, we
could utilize the channel num info to optimize searh param in queryHook

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-31 18:59:49 +08:00
wayblink
ce3f836876
fix: compaction task not be cleaned correctly (#34765) 
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction

bug: #34764

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-30 20:21:56 +08:00
yihao.dai
39d25938c5
enhance: Avoid panic due to nil schema (#35063) 
/kind improvement

issue: https://github.com/milvus-io/milvus/discussions/25620

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-30 20:18:32 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
congqixia
972752258a
enhance: Support otlp http exporter (#35053) 
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 17:43:49 +08:00
SimFG
0e41f104c5
enhance: use the collection id to group msg pack in the msg dispatcher (#34871) 
- issue: #34870

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-29 11:45:49 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
yihao.dai
4a3d98d88c
enhance: Fast close msgdispatcher target (#34803) 
/kind improvement

issue: https://github.com/milvus-io/milvus/issues/34075

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-29 00:25:49 +08:00
PowderLi
5584ae2e14
enhance: add metrics for restful apis (#34969) 
issue: #34968

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-07-29 00:13:47 +08:00
cai.zhang
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time (#34949) 
issue: #34703

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-28 23:53:47 +08:00
zhagnlu
804ec24c02
fix:fix retrieve raw data from bitmap array index (#34848) 
#34795

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-27 01:53:47 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs (#34763) 
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 10:01:46 +08:00
Ted Xu
63f0154dfb
fix: enable milvus.yaml check (#34567) 
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-07-25 18:05:46 +08:00