tinswzy
c7f21d5a06
enhance: purge small files right after wp segment compaction ( #44473 )
...
#43638
improve wp log output
[wp#43](https://github.com/zilliztech/woodpecker/issues/43 )
intro purge small files right after segment compaction
[wp#47](https://github.com/zilliztech/woodpecker/issues/47 )
The rootpath configured by milvus is uniformly used as the base for wp
local fs storage.
update to v0.1.5
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-09-21 16:32:01 +08:00
yihao.dai
51f69f32d0
feat: Add CDC support ( #44124 )
...
This PR implements a new CDC service for Milvus 2.6, providing log-based
cross-cluster replication.
issue: https://github.com/milvus-io/milvus/issues/44123
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: chyezh <chyezh@outlook.com>
2025-09-16 16:32:01 +08:00
tinswzy
6a342edc5a
fix: empty error returned on append timeout when MinIO is unavailable ( #43926 )
...
#43810
Fixed the issue where the result err returned by append timeout was
empty when objectstorage was unavailable, causing the client to
mistakenly believe that the write was successful.
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-08-19 10:47:45 +08:00
tinswzy
084f777552
enhance: use wp internal writer without lock ( #43775 )
...
#43638 #43810
add internal writer without session lock;
refactor and unify read state and log entry
refactor data reading related methods;
fix bug where a closed writer is reused for finalize;
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-08-18 01:15:44 +08:00
Zhen Ye
8ff118a9ff
fix: call IntoMessageProto instead of Payload when rpc ( #43678 )
...
issue: #43677
Signed-off-by: chyezh <chyezh@outlook.com>
2025-08-06 14:45:40 +08:00
Zhen Ye
3e3775fb81
fix: panics when describe collection internal failure ( #43630 )
...
issue: #43629
- also fix the scanner_switchable panic underlying wal scanner return
context error.
Signed-off-by: chyezh <chyezh@outlook.com>
2025-07-29 20:33:36 +08:00
tinswzy
173efe2b98
enhance: wp metrics and update deps to v0.1.0 ( #43569 )
...
#43574 #43604 #43431 #43603
Fix wp metrics not registered bug;
Update the version dependent on wp to v0.1.2-rc1;
improve advanced reader with concurrent prefetch blks;
add the segment rolling policy based on the number of blocks;
improve concurrent compaction
release lock failed bug
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-07-29 14:51:35 +08:00
tinswzy
7da62698e0
enhance: improve WP parallel sync mechanism and fencing logic ( #42892 )
...
related: #42595
improve WP parallel sync mechanism and fencing logic; remove redundant
metrics and labels
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-07-13 23:04:49 +08:00
tinswzy
36a4b74fc0
enhance: Adjust the default parameters of WP according to performance tests ( #42598 )
...
#42595
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-06-10 16:30:35 +08:00
tinswzy
f55f900c85
fix insert hang caused by WAL writer writing to a closing logfile ( #42078 )
...
related issue #42049
wp commit
[94de4](94de4cbc60
2025-06-03 09:58:36 +08:00
tinswzy
1735f557ca
fix sn oom issue during small file loading in wp ( #41946 )
...
#41846 #41894
Resolve SN OOM issue during small file loading in Woodpecker;
Correct WP fence/close execution order;
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-05-23 01:30:28 +08:00
tinswzy
4edb1bc6f1
fix: resolve wp WALImpls concurrent read/write bug ( #41763 )
...
#41563 #41579 #41842 #41846 #41758
Upgraded the wp dependency to incorporate recent fixes addressing
multiple concurrency bugs in WALImpls.
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-05-16 12:02:27 +08:00
Zhen Ye
52950ce392
enhance: add pulsar truncate api to protect pulsar unconsumed message ( #41724 )
...
issue: #41465
- implement truncate api for pulsar based on durable subscription.
- truncate api can only be called if wal is read-write.
Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-11 20:50:55 +08:00
tinswzy
b36ed03141
enhance: Add Truncate Interface to WALImpls for Log Retention Control ( #41517 )
...
#41465 Add Truncate Interface to WALImpls for Proactive Log Retention
Management
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-29 14:36:49 +08:00
Zhen Ye
dfbb02a5f7
enhance: make streaming message as a log field for easier coding ( #41545 )
...
issue: #41544
- implement message can be logged as a field by zap.
- fix too many slow log for woodpecker.
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-28 14:38:42 +08:00
tinswzy
6fa68c1f16
enhance: Support Woodpecker as a WAL storage option for Milvus ( #41095 )
...
#40916 Support Woodpecker as a WAL storage option for Milvus
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-20 22:22:42 +08:00
