admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-06 17:18:35 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,557 Commits 133 Branches 183 Tags
Commit Graph

62 Commits

Author SHA1 Message Date
Chun Han
001619aef9
feat: supporing load priority for loading (#42413) 
related: #40781

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-06-17 15:22:38 +08:00
Bingyi Sun
cc5ac1c220
enhance: Support cast function for json index (#41949) 
issue: #41948

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-06-05 19:42:32 +08:00
groot
1574673a8c
enhance: Alter collection description (#41558) 
issue: https://github.com/milvus-io/milvus/issues/41557

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2025-05-12 14:16:55 +08:00
cai.zhang
640f526301
fix: Update current scalar index version to compatible tantivy different versions (#41141) 
issue: #40823

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-04-27 20:44:39 +08:00
SimFG
91d40fa558
fix: Update logging context and upgrade dependencies (#41318) 
- issue: #41291

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-04-23 10:52:38 +08:00
Chun Han
016920b023
fix: solve incompitable problem for none-encoding index(#40838) (#41369) 
related: #40838

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-20 22:56:44 +08:00
Xianhui Lin
3bc24c264f
enhance: Add json key inverted index in stats for optimization (#38039) 
Add json key inverted index in stats for optimization
https://github.com/milvus-io/milvus/issues/36995

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-04-10 15:20:28 +08:00
smellthemoon
cb1e86e17c
enhance: support add field (#39800) 
after the pr merged, we can support to insert, upsert, build index,
query, search in the added field.
can only do the above operates in added field after add field request
complete, which is a sync operate.

compact will be supported in the next pr.
#39718

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-04-02 14:24:31 +08:00
SimFG
16efcda5c4
feat: support to deny dll according to database property (#40764) 
- issue: #40762

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-23 11:18:28 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990) 
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 23:15:58 +08:00
Bingyi Sun
b59555057d
feat: support json index (#36750) 
https://github.com/milvus-io/milvus/issues/35528

This PR adds json index support for json and dynamic fields. Now you can
only do unary query like 'a["b"] > 1' using this index. We will support
more filter type later.

basic usage:
```
collection.create_index("json_field", {"index_type": "INVERTED",
    "params": {"json_cast_type": DataType.STRING, "json_path":
'json_field["a"]["b"]'}})
```

There are some limits to use this index:
1. If a record does not have the json path you specify, it will be
ignored and there will not be an error.
2. If a value of the json path fails to be cast to the type you specify,
it will be ignored and there will not be an error.
3. A specific json path can have only one json index.
4. If you try to create more than one json indexes for one json field,
sdk(pymilvus<=2.4.7) may return immediately because of internal
implementation. This will be fixed in a later version.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-02-15 14:06:15 +08:00
SimFG
f070af67f7
fix: deny to set the mmap param for the alter index api when enable auto index (#39518) 
- issue: #39517

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-01-23 10:01:04 +08:00
cai.zhang
6d45dd5666
fix: Add scalar index engine version for compatibility (#39204) 
issue: #39203

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-01-15 12:25:00 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg (#37559) 
- issue: #37105

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-12-17 09:08:46 +08:00
Gao
8977454311
enhance: support recall estimation (#38017) 
issue: #37899 
Only `search` api will be supported

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-12-11 20:40:48 +08:00
Gao
994fc544e7
enhance: support iterative filter execution (#37363) 
issue: #37360

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-12-11 11:32:44 +08:00
congqixia
24c6a4bb29
enhance: [GoSDK] support Doc-in-doc-out APIs (#37590) 
Related to #35853

This PR contains following changes:

- Add function and related proto and helper functions
- Remove the insert column missing check and leave it to server
- Add text as search input data
- Add some unit tests for logic above

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-12 10:58:28 +08:00
smellthemoon
6e880d19a8
enhance: add search params in search request in restful (#36304) 
https://github.com/milvus-io/milvus/issues/36321

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-23 16:51:12 +08:00
aoiasd
139787371e
feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036) 
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-19 10:57:12 +08:00
aoiasd
da227ff9a1
feat: Support create collection with functions (#35973) 
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-12 10:43:06 +08:00
cai.zhang
2c9bb4dfa3
feat: Support stats task to sort segment by PK (#35054) 
issue: #33744 

This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-09-02 14:19:03 +08:00
jaime
b0ac04d104
enhance: set database properties to restrict read access (#35745) 
issue: #35744

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-29 13:17:01 +08:00
zhagnlu
42f7800b5b
enhance: add bitmap offset cache to speed up retrieve raw data (#35498) 
#35458

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-24 01:40:58 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359) 
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-21 00:22:54 +08:00
congqixia
2fbc628994
feat: Support field partial load collection (#35416) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-20 16:49:02 +08:00
wei liu
853d2f33e0
enhance: Trim space during parsing string list from string (#35546) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 20:48:55 +08:00
Gao
6695c6d0a3
enhance: add channel num for queryHook optimization (#35104) 
At most cases, data in each channel is almost evenly distributed, we
could utilize the channel num info to optimize searh param in queryHook

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-31 18:59:49 +08:00
zhagnlu
f1b2f7b640
enhance: refactor bitmap index and internal hybrid index (#34450) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-18 10:39:42 +08:00
SimFG
d8e68cbb97
enhance: use the insert rate limit for the upsert request (#34615) 
- issue: #34614

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-12 16:17:35 +08:00
Patrick Weizhi Xu
104d0966b7
feat: support partition key isolation (#34336) 
issue: #34332

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-11 19:01:35 +08:00
wei liu
9b37d3f517
enhance: Enable setting the replica number and resource group during collection creation (#34403) 
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-10 10:20:13 +08:00
Gao
ae6d6f91e6
enhance: change autoindex default metric type (#34261) 
issue: #34304 
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-08 19:52:24 +08:00
jaime
60be454db0
enhance: add disk quota and max collections into db properties (#34368) 
issue: #34385

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 18:22:17 +08:00
Buqian Zheng
fa8d641ce6
fix: update check for sparse hnsw index (#33713) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-07-02 21:56:09 +08:00
Gao
a789c60380
enhance: autoindex for multi data type (#33868) 
issue: #22837 

contain https://github.com/milvus-io/milvus/pull/33625
https://github.com/milvus-io/milvus/pull/33867
https://github.com/milvus-io/milvus/pull/33911 which already merged to
2.4 branch

Signed-off-by: chasingegg <chao.gao@zilliz.com>
Co-authored-by: foxspy <xianliang.li@zilliz.com>
 2024-06-18 21:34:01 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data (#33651) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
 2024-06-06 17:37:51 +08:00
zhagnlu
589d4dfd82
enhance: optimize bitmap index (#33358) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-05-30 13:09:43 +08:00
wei liu
b13932bb55
enhance: Enable database level replica num and resource groups for loading collection (#33052) 
issue: #30040

This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups

User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-29 10:59:43 +08:00
SimFG
4031abd2fa
enhance: change default partition num to 16 when using partition key (#32950) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-13 14:19:31 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels (#31070) 
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-04-12 16:01:19 +08:00
Buqian Zheng
2fdf1a6e76
feat: [Sparse Float Vector] added some integration tests (#31062) 
add some integration tests for sparse float vector support

https://github.com/milvus-io/milvus/issues/29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-04-10 19:57:18 +08:00
Bingyi Sun
fbff46a005
enhance: add lazyload global config (#31610) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-27 20:23:10 +08:00
Chun Han
c3264ca3e3
feat: support segment pruner (#31003) 
related: #30376
 2024-03-22 13:57:06 +08:00
congqixia
a647b84f3e
enhance: Add AllPartitionsID const to replace InvalidPartitionID (#31438) 
"-1" as `InvalidPartitionID` previously used as All partition place
holder in delete cases. It's confusing and hard to maintain when a const
var has more than one meaning.

This PR add `AllPartitionsID` to replace these usages in delete
scenarios.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-20 19:01:05 +08:00
Bingyi Sun
bdc70dfc6a
feat: Add global mmap enable configuration (#31267) 
https://github.com/milvus-io/milvus/issues/31279

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-18 15:17:10 +08:00
Bingyi Sun
fd17a5f050
fix: check collection lazy load prop using schema (#30992) 
issue: https://github.com/milvus-io/milvus/issues/30361

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-06 16:19:01 +08:00
Bingyi Sun
7783098ddd
feat: support lazy load on querycoord (#30372) 
https://github.com/milvus-io/milvus/issues/30361

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-01 18:15:29 +08:00
Gao
8a630f733a
enhance: add new optimize param for queryhook (#29495) 
add a flag to indicate if we use search param optimizations, default is
on

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-12-28 10:04:46 +08:00
yah01
dc89730a50
Support collection-level mmap control (#26901) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-02 23:52:16 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00