14 Commits

Author SHA1 Message Date
tinswzy
6a342edc5a
fix: empty error returned on append timeout when MinIO is unavailable (#43926)
#43810 
Fixed the issue where the result err returned by append timeout was
empty when objectstorage was unavailable, causing the client to
mistakenly believe that the write was successful.

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-08-19 10:47:45 +08:00
tinswzy
084f777552
enhance: use wp internal writer without lock (#43775)
#43638  #43810 
add internal writer without session lock;
refactor and unify read state and log entry
refactor data reading related methods;
fix bug where a closed writer is reused for finalize;

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-08-18 01:15:44 +08:00
Zhen Ye
8ff118a9ff
fix: call IntoMessageProto instead of Payload when rpc (#43678)
issue: #43677

Signed-off-by: chyezh <chyezh@outlook.com>
2025-08-06 14:45:40 +08:00
Zhen Ye
3e3775fb81
fix: panics when describe collection internal failure (#43630)
issue: #43629

- also fix the scanner_switchable panic underlying wal scanner return
context error.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-07-29 20:33:36 +08:00
tinswzy
173efe2b98
enhance: wp metrics and update deps to v0.1.0 (#43569)
#43574   #43604 #43431  #43603 
Fix wp metrics not registered bug;
Update the version dependent on wp to v0.1.2-rc1;
improve advanced reader with concurrent prefetch blks;
add the segment rolling policy based on the number of blocks;
improve concurrent compaction
release lock failed bug

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-07-29 14:51:35 +08:00
tinswzy
7da62698e0
enhance: improve WP parallel sync mechanism and fencing logic (#42892)
related: #42595 
improve WP parallel sync mechanism and fencing logic; remove redundant
metrics and labels

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-07-13 23:04:49 +08:00
tinswzy
36a4b74fc0
enhance: Adjust the default parameters of WP according to performance tests (#42598)
#42595

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-06-10 16:30:35 +08:00
tinswzy
f55f900c85
fix insert hang caused by WAL writer writing to a closing logfile (#42078)
related issue #42049 
wp commit
[94de4](94de4cbc60)

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-06-03 09:58:36 +08:00
tinswzy
1735f557ca
fix sn oom issue during small file loading in wp (#41946)
#41846  #41894 
Resolve SN OOM issue during small file loading in Woodpecker; 
Correct WP fence/close execution order;

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-05-23 01:30:28 +08:00
tinswzy
4edb1bc6f1
fix: resolve wp WALImpls concurrent read/write bug (#41763)
#41563 #41579 #41842 #41846 #41758
Upgraded the wp dependency to incorporate recent fixes addressing
multiple concurrency bugs in WALImpls.

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-05-16 12:02:27 +08:00
Zhen Ye
52950ce392
enhance: add pulsar truncate api to protect pulsar unconsumed message (#41724)
issue: #41465

- implement truncate api for pulsar based on durable subscription.
- truncate api can only be called if wal is read-write.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-11 20:50:55 +08:00
tinswzy
b36ed03141
enhance: Add Truncate Interface to WALImpls for Log Retention Control (#41517)
#41465 Add Truncate Interface to WALImpls for Proactive Log Retention
Management

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-29 14:36:49 +08:00
Zhen Ye
dfbb02a5f7
enhance: make streaming message as a log field for easier coding (#41545)
issue: #41544

- implement message can be logged as a field by zap.
- fix too many slow log for woodpecker.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-28 14:38:42 +08:00
tinswzy
6fa68c1f16
enhance: Support Woodpecker as a WAL storage option for Milvus (#41095)
#40916 Support Woodpecker as a WAL storage option for Milvus

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-20 22:22:42 +08:00