cai.zhang
8a77fb9cdc
enhance: Support slot for index task and stats task ( #39084 )
...
issue: #39101
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-08 20:46:25 +08:00
congqixia
1953676ee0
fix: Set task queue time before enqueue ( #40815 )
...
Related to #40814
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-21 19:36:15 +08:00
cai.zhang
6dbe5d475e
enhance: Refine task meta with key lock ( #40613 )
...
issue: #39101
2.5 pr: #40146 #40353
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-14 15:44:22 +08:00
cai.zhang
5a810400b5
enhance: Optimize Task Scheduling to Enable Concurrent Execution ( #40251 )
...
issue: #39101
2.5 pr: #40104
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-02 18:38:00 +08:00
cai.zhang
762a644d76
enhance: Limit the speed of the generating stats task ( #39644 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-28 10:27:59 +08:00
cai.zhang
9f5b488f9a
enhance: Export request timeout interval in config ( #40119 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-23 15:15:54 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
jaime
8a4ac8cccd
enhance: expose more metrics data ( #39456 )
...
issue: #36621 #39417
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.
Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-02-07 11:50:50 +08:00
cai.zhang
1d54ff157f
fix: Restore the compacting state for stats task during recovery ( #39459 )
...
issue: #39333
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2025-02-05 17:11:12 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto ( #39114 )
...
issue: #39095
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
cai.zhang
9be106dedf
enhance: Refine task scheduler logs ( #38334 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-11 15:00:44 +08:00
yihao.dai
b6612e02b4
enhance: Reduce GetIndexInfos calls ( #37695 )
...
Batch `GetIndexInfos` calls for segments to reduce RPC calls.
issue: https://github.com/milvus-io/milvus/issues/37634
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-19 14:24:31 +08:00
jaime
1e8ea4a7e7
feat: add segment/channel/task/slow query render ( #37561 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-12 17:44:29 +08:00
jaime
9d16b972ea
feat: add tasks page into management WebUI ( #37002 )
...
issue: #36621
1. Add API to access task runtime metrics, including:
- build index task
- compaction task
- import task
- balance (including load/release of segments/channels and some leader
tasks on querycoord)
- sync task
2. Add a debug model to the webpage by using debug=true or debug=false
in the URL query parameters to enable or disable debug mode.
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-28 10:13:29 +08:00
cai.zhang
7bf40694fd
fix: Record the nodeID before assigning tasks ( #36371 )
...
issue: #33744
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-28 17:21:15 +08:00
cai.zhang
8395c8a8db
enhance: Update stats task to optional ( #35947 )
...
issue: #33744
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-12 20:37:08 +08:00
cai.zhang
2c9bb4dfa3
feat: Support stats task to sort segment by PK ( #35054 )
...
issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
congqixia
582d2eec79
enhance: Move datanode/indexnode manager to session pkg ( #35634 )
...
Related to #28861
Move session manager, worker manager to session package. Also renaming
each manager to corresponding node name(datanode, indexnode).
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 16:02:56 +08:00
cai.zhang
6542c1ab0e
enhance: Add monitoring metrics for task execution time in datacoord ( #35139 )
...
issue: #35138
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-05 16:26:17 +08:00
cai.zhang
feb13cdf07
enhance: Skip pick worker when task doesn't need to execute actually ( #34348 )
...
issue: #34347
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-03 15:52:09 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data ( #33651 )
...
issue: #30633
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
2024-06-06 17:37:51 +08:00
