15 Commits

Author SHA1 Message Date
yihao.dai
b18ebd9468
enhance: Remove legacy cdc/replication (#46603)
issue: https://github.com/milvus-io/milvus/issues/44123

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: legacy in-cluster CDC/replication plumbing
(ReplicateMsg types, ReplicateID-based guards and flags) is obsolete —
the system relies on standard msgstream positions, subPos/end-ts
semantics and timetick ordering as the single source of truth for
message ordering and skipping, so replication-specific
channels/types/guards can be removed safely.

- Removed/simplified logic (what and why): removed replication feature
flags and params (ReplicateMsgChannel, TTMsgEnabled,
CollectionReplicateEnable), ReplicateMsg type and its tests, ReplicateID
constants/helpers and MergeProperties hooks, ReplicateConfig and its
propagation (streamPipeline, StreamConfig, dispatcher, target),
replicate-aware dispatcher/pipeline branches, and replicate-mode
pre-checks/timestamp-allocation in proxy tasks — these implemented a
redundant alternate “replicate-mode” pathway that duplicated
position/end-ts and timetick logic.

- Why this does NOT cause data loss or regression (concrete code paths):
no persistence or core write paths were removed — proxy PreExecute flows
(internal/proxy/task_*.go) still perform the same schema/ID/size
validations and then follow the normal non-replicate execution path;
dispatcher and pipeline continue to use position/subPos and
pullback/end-ts in Seek/grouping (pkg/mq/msgdispatcher/dispatcher.go,
internal/util/pipeline/stream_pipeline.go), so skipping and ordering
behavior remains unchanged; timetick emission in rootcoord
(sendMinDdlTsAsTt) is now ungated (no silent suppression), preserving or
increasing timetick delivery rather than removing it.

- PR type and net effect: Enhancement/Refactor — removes deprecated
replication API surface (types, helpers, config, tests) and replication
branches, simplifies public APIs and constructor signatures, and reduces
surface area for future maintenance while keeping DML/DDL persistence,
ordering, and seek semantics intact.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-12-30 14:53:21 +08:00
aoiasd
201e980d3d
fix: flow graph should free function resource after all node close (#42731)
relate: https://github.com/milvus-io/milvus/issues/42730

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-06-13 22:14:37 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg (#37559)
- issue: #37105

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
congqixia
10460ed3f8
enhance: Simplify querynode tsafe & reduce goroutine number (#38416)
Related to #37630

TSafe manager is too complex for current implementation and each
delegator need one goroutine waiting for tsafe update event.

Tsafe updating could be executed in pipeline. This PR remove tsafe
manager and simplify the entire logic of tsafe updating.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-13 10:56:43 +08:00
jaime
f348bd9441
feat: add segment,pipeline, replica and resourcegroup api for WebUI (#37344)
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-07 11:52:25 +08:00
aoiasd
db34572c56
feat: support load and query with bm25 metric (#36071)
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-11 10:23:20 +08:00
congqixia
efa0c73c62
fix: Unify querynode metrics cleanup in collection release (#32805)
Related to #32803

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 15:41:29 +08:00
wei liu
df208d538c
fix: Check exclude segment before add new growing segment (#31803)
issue: #31479 #31797

milvus will add released segment to excluded info, and filter out it's
stream data in filter_node. but for data buffered in insert_node's
channel, if it belongs to growing segment which already be released,
then it will all the growing segment back again.

This PR maintain `excluded segments` in delegator, and check excluded
segment before new growing segment.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-10 15:29:17 +08:00
aoiasd
3633923bb7
enhance: clean invalid pipline excluded segment info (#30429)
relate: https://github.com/milvus-io/milvus/issues/30281

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 10:43:01 +08:00
congqixia
a8b7629315
fix: exclude insertData before growing checkpoint (#29558)
Resolves: #29556
Refine exclude segment function signature
Add exclude growing before checkpoint logic

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-28 18:18:54 +08:00
Enwei Jiao
d3af451d92
Upgrade golangci-lint (#24707)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 19:34:36 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
aoiasd
970dcb3161
Fix panic because timestamp was nil (#23096)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-29 17:20:04 +08:00
yah01
081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00