Buqian Zheng
75557f3eb8
enhance: Use std::shared_lock and std::unique_lock for mutexes ( #44459 )
...
issue: https://github.com/milvus-io/milvus/issues/44452
Signed-off-by: zhengbuqian <zhengbuqian@gmail.com>
Co-authored-by: buqian.zheng <buqian.zheng@zilliz.com>
2025-09-22 18:02:09 +08:00
zhagnlu
8934c18792
enhance: support cache result cache for expr ( #43923 )
...
issue: #43878
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-08-26 10:55:52 +08:00
sparknack
4fae074d56
enhance: add write rate limit for disk file writer ( #43912 )
...
issue: #43040
---------
Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
2025-08-25 10:27:47 +08:00
Bingyi Sun
b59bc5e2c0
fix: make json path index non exists offsets compatible with 2.5 ( #43691 )
...
issue: https://github.com/milvus-io/milvus/issues/43666
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-08-01 23:22:23 +08:00
Bingyi Sun
742d72a6c2
fix: Fix wrong null offsets for json path index ( #43390 )
...
issue: https://github.com/milvus-io/milvus/issues/43315
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-07-26 17:26:54 +08:00
Spade A
10fe53ff59
feat: support json for ngram ( #43170 )
...
Ref https://github.com/milvus-io/milvus/issues/42053
This PR enable ngram to support json data type.
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-07-25 10:28:54 +08:00
Spade A
fce0bbe2ae
fix: remove redundant locks for null_offset ( #43103 )
...
Ref: https://github.com/milvus-io/milvus/issues/40308
https://github.com/milvus-io/milvus/pull/40363 add lock for protecting
concurrent read/write for null offset. But we don't need this for sealed
segment.
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-07-04 10:10:45 +08:00
Bingyi Sun
669ea51ce5
enhance: Make json index compatible with caching layer ( #42484 )
...
issue: https://github.com/milvus-io/milvus/issues/42483
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-06-24 15:16:41 +08:00
Bingyi Sun
fbf5cb4e62
feat: Add json flat index ( #39917 )
...
issue: https://github.com/milvus-io/milvus/issues/35528
This PR introduces a JSON flat index that allows indexing JSON fields
and dynamic fields in the same way as other field types.
In a previous PR (#36750 ), we implemented a JSON index that requires
specifying a JSON path and casting a type. The only distinction lies in
the json_cast_type parameter. When json_cast_type is set to JSON type,
Milvus automatically creates a JSON flat index.
For details on how Tantivy interprets JSON data, refer to the [tantivy
documentation](https://github.com/quickwit-oss/tantivy/blob/main/doc/src/json.md#pitfalls-limitation-and-corner-cases ).
Limitations
Array handling: Arrays do not function as nested objects. See the
[limitations
section](https://github.com/quickwit-oss/tantivy/blob/main/doc/src/json.md#arrays-do-not-work-like-nested-object )
for more details.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-06-10 19:14:35 +08:00
Bingyi Sun
6404e02d99
fix: Check cast type is array for json contains expr ( #42184 )
...
issue: https://github.com/milvus-io/milvus/issues/42181
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-06-09 17:04:33 +08:00
Bingyi Sun
cc5ac1c220
enhance: Support cast function for json index ( #41949 )
...
issue: #41948
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-06-05 19:42:32 +08:00
junjiejiangjjj
0b2ecb7632
fix: Solve clang compilation errors ( #42041 )
...
https://github.com/milvus-io/milvus/issues/42040
Signed-off-by: junjiejiangjjj <junjie.jiang@zilliz.com>
2025-05-27 20:32:29 +08:00
Bingyi Sun
4c08090687
feat: Add json index support for json contains expr ( #41478 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-05-06 11:44:52 +08:00
Bingyi Sun
355f62d6c9
fix: Align brute force search with json index for exists expr ( #41116 )
...
issue: #35528
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-07 15:42:23 +08:00
Bingyi Sun
fcb03b5bd1
feat: add json null/exists expression ( #41004 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-03 17:48:21 +08:00
Bingyi Sun
27ff3a42e7
enhance: Record simdjson error ( #41003 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-31 17:56:19 +08:00
Bingyi Sun
9676365af9
fix: Fix json index not equal filter ( #40647 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-27 23:06:23 +08:00
Bingyi Sun
6249335859
fix: Catch invalid json pointer error ( #40625 )
...
issue: #35528
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-14 16:56:08 +08:00
Spade A
3db56560fb
fix: fix concurrent issues in null offset ( #40363 )
...
issue: #40308
This issue fixes these two concurrent issues:
1. element in null_offset is used to set bitset where the size of bitset
is initialized by tantivy document count. However, there may still be
some documents that are not committed in tantivy but are null in
null_offset. So array out of range occurs.
2. null_offset can be read and write concurrently but there's no
synchronization protection.
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-03-05 17:48:00 +08:00
Bingyi Sun
be4d09561b
fix: Fix missing null or non-exist key in json index ( #40336 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-05 11:48:02 +08:00
Bingyi Sun
b59555057d
feat: support json index ( #36750 )
...
https://github.com/milvus-io/milvus/issues/35528
This PR adds json index support for json and dynamic fields. Now you can
only do unary query like 'a["b"] > 1' using this index. We will support
more filter type later.
basic usage:
```
collection.create_index("json_field", {"index_type": "INVERTED",
"params": {"json_cast_type": DataType.STRING, "json_path":
'json_field["a"]["b"]'}})
```
There are some limits to use this index:
1. If a record does not have the json path you specify, it will be
ignored and there will not be an error.
2. If a value of the json path fails to be cast to the type you specify,
it will be ignored and there will not be an error.
3. A specific json path can have only one json index.
4. If you try to create more than one json indexes for one json field,
sdk(pymilvus<=2.4.7) may return immediately because of internal
implementation. This will be fixed in a later version.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-02-15 14:06:15 +08:00
