admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-29 06:55:27 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,660 Commits 136 Branches 185 Tags
Commit Graph

621 Commits

Author SHA1 Message Date
wei liu
32e55a02ea
fix: Fix privilege group hasn't been register for validate (#35937) 
issue: #35471

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-05 15:35:04 +08:00
congqixia
8593c4580a
enhance: Add delete buffer related quota logic (#35918) 
See also #35303

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-05 11:39:03 +08:00
jaime
24fb10114b
enhance: remove cooling off in rate limiter for read requests (#35935) 
issue: #35934

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-04 14:39:10 +08:00
zhagnlu
325f1987d9
enhance: rewrite index params for compatibility (#35788) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-02 19:41:03 +08:00
cai.zhang
2c9bb4dfa3
feat: Support stats task to sort segment by PK (#35054) 
issue: #33744 

This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-09-02 14:19:03 +08:00
congqixia
9d80137698
fix: Check clustering key skip load behavior (#35865) 
feature issue: #35415
See also #35861

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-02 11:17:02 +08:00
SimFG
311f860676
enhance: support to drop the role which is related the privilege list (#35727) 
- issue: #35545

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-30 15:17:00 +08:00
jaime
b0ac04d104
enhance: set database properties to restrict read access (#35745) 
issue: #35744

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-29 13:17:01 +08:00
zhagnlu
4d2f96c760
enhance: support bitmap mmap (#35399) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-27 16:34:59 +08:00
smellthemoon
098c10922b
enhance: add some check when create collection (#35629) 
not support nullable==true in vector type. check it when create
collection.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-25 16:23:05 +08:00
Xiaofan
50fcfe8ef1
enhance: add nan and inf check (#35683) 
fix #35594
add float check on files

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-25 15:22:57 +08:00
zhagnlu
42f7800b5b
enhance: add bitmap offset cache to speed up retrieve raw data (#35498) 
#35458

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-24 01:40:58 +08:00
smellthemoon
ba6db117e3
enhance: add some integration test in null (#35599) 
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-21 17:44:56 +08:00
wei liu
cf242f9e09
fix: fix dynamic update config doesn't works for some param (#35572) 
issue: #35570
milvus support config cache to spped up config access, but only evict
param's cache when param has been updated. but milvus's param may rely
on other param's value, let's say ParamsA relys on paramsB, when paramsB
updated, it will evict paramB's cache, but the paramA's cache still keep
the old value.

This PR evict all config cache to solve the above issue, cause dynamic
update config won't be much frequetly.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-21 11:02:56 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359) 
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-21 00:22:54 +08:00
Ted Xu
41646c8439
feat: integrate new deltalog format (#35522) 
See #34123

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-20 19:06:56 +08:00
congqixia
2fbc628994
feat: Support field partial load collection (#35416) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-20 16:49:02 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:16:55 +08:00
Zhen Ye
4d69898cb2
enhance: support single pchannel level transaction (#35289) 
issue: #33285

- support transaction on single wal.
- last confirmed message id can still be used when enable transaction.
- add fence operation for segment allocation interceptor.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-19 21:22:56 +08:00
wei liu
853d2f33e0
enhance: Trim space during parsing string list from string (#35546) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 20:48:55 +08:00
wei liu
ead37326f0
enhance: Add backup/restore privilege to admin group (#35515) 
issue: #35443

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 18:42:54 +08:00
wei liu
a570567644
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) 
issue: #35471

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 14:18:54 +08:00
Buqian Zheng
f4a91e135b
enhance: Allow empty sparse row (#34700) 
issue: #29419

* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-08-16 14:14:54 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 10:10:53 +08:00
Ted Xu
c6ae7d4d56
enhance: adding the msgchannel section in generated yaml (#35466) 
See #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-14 18:32:53 +08:00
congqixia
f99a761d87
enhance: Support dynamic field in SchemaHelper (#35461) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-14 18:04:59 +08:00
Reid Chan
6c135e5ea9
fix: key of DataCoordTicklePath is wrong (issue: #35259) (#35263) 
issue: [35259](https://github.com/milvus-io/milvus/issues/35259)

Signed-off-by: Reid Chan <reidchan@apache.org>
 2024-08-13 17:40:20 +08:00
smellthemoon
c4a240acb1
enhance: memory value are displayed in MB (#35292) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-09 14:14:25 +08:00
wei liu
344dc6a9f8
enhance: enable to set load config in cluster level (#35169) 
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-07 12:38:21 +08:00
Xiaofan
885d891106
Change minio use virtual config to not panic (#35327) 
minio use virtual should not panic if empty. User may use other object
storage instead

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-07 09:58:15 +08:00
presburger
a9352a073b
enhance: change gpu default mem pool size (#34159) 
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-08-06 10:24:15 +08:00
cai.zhang
6542c1ab0e
enhance: Add monitoring metrics for task execution time in datacoord (#35139) 
issue: #35138

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-05 16:26:17 +08:00
congqixia
a6903701a4
enhance: Remove stackerror imported unexpected (#35256) 
`stackerror` pkg is imported by accident and could be replaced by
cockroachdb errors lib. This PR removes import line and fix this
problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-05 14:48:16 +08:00
chyezh
9871966415
enhance: segment alloc interceptor (#34996) 
#33285

- add segment alloc interceptor for streamingnode.
- add add manual alloc segment rpc for datacoord.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-04 07:40:15 +08:00
yihao.dai
a4439cc911
enhance: Implement flusher in streamingNode (#34942) 
- Implement flusher to:
  - Manage the pipelines (creation, deletion, etc.)
  - Manage the segment write buffer
  - Manage sync operation (including receive flushMsg and execute flush)
- Add a new `GetChannelRecoveryInfo` RPC in DataCoord.
- Reorganize packages: `flushcommon` and `datanode`.

issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-02 18:30:23 +08:00
jaime
fcec4c21b9
fix: check collection health(queryable) fail for releasing collection (#34947) 
issue: #34946

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-02 17:20:15 +08:00
congqixia
c64a078458
enhance: Support proxy/delegator qn client pooling (#35194) 
See also #35196

Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
wei liu
3b735b4b02
enhance: Refine param init for MmapDirPath (#35181) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 11:12:14 +08:00
cai.zhang
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size (#35076) 
issue: #34495

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 22:04:14 +08:00
Ted Xu
b4d0f4df0a
enhance: update param documents (#35152) 
See: #35161

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-01 12:38:13 +08:00
wei liu
e9d61daa3f
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 10:21:50 +08:00
wayblink
ce3f836876
fix: compaction task not be cleaned correctly (#34765) 
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction

bug: #34764

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-30 20:21:56 +08:00
yihao.dai
39d25938c5
enhance: Avoid panic due to nil schema (#35063) 
/kind improvement

issue: https://github.com/milvus-io/milvus/discussions/25620

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-30 20:18:32 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
congqixia
972752258a
enhance: Support otlp http exporter (#35053) 
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 17:43:49 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
cai.zhang
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time (#34949) 
issue: #34703

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-28 23:53:47 +08:00
zhagnlu
804ec24c02
fix:fix retrieve raw data from bitmap array index (#34848) 
#34795

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-27 01:53:47 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs (#34763) 
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 10:01:46 +08:00
Ted Xu
63f0154dfb
fix: enable milvus.yaml check (#34567) 
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-07-25 18:05:46 +08:00