273 Commits

Author SHA1 Message Date
wei liu
336378c198
update shard leader cache (#22632)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-09 16:35:53 +08:00
aoiasd
69f4afe456
Fix data node tickle not run default (#22622)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-08 15:23:51 +08:00
xige-16
72264f656f
Fix concurrent loading diskann index timeout (#22548)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-03-03 20:15:49 +08:00
Ten Thousand Leaves
9129c11dec
Fix enableAutoCompaction default value (#22255)
/kind bug

issue: #22253

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-03-02 14:05:48 +08:00
MrPresent-Han
b01b467bf9
support channel balancer on datacoord (#22324) (#22377)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-02-27 18:27:48 +08:00
Xiaofan
ea039bbd37
Change the etcd session timeout to 20s (#22400)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-27 18:11:48 +08:00
wayblink
4cf8f2ec81
Add memory usage too large sync policy (#22241)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-27 17:47:51 +08:00
SimFG
8123161738
Add the retry op for all server (#22274)
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
2023-02-24 17:51:47 +08:00
SimFG
81f78142ee
Only print the file content when log the hook.yaml (#22340)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-02-22 20:55:46 +08:00
wei liu
cd2e7fa535
sync resource group change (#22269)
fix load collection with rg (#22083)
	refine rg capacity behavior (#22089)
	support rg api rbac (#22097)
	Add resource group tests (#22101)
	Fix gen_vectors typo issue (#22117)
	fix transfer node (#22120)
	fix rg e2e (#22187)

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-20 19:22:26 +08:00
aoiasd
fbbbd6ea2f
Add tickle for datacoord watch event (#21193) (#22209)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-17 15:52:36 +08:00
SimFG
bd559b3148
Add error log and metrics for the hook and privilege interceptor (#22111)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-02-10 15:52:32 +08:00
wayblink
d63bf18236
Support print log to file and console at the same time (#22080)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-10 14:16:32 +08:00
Xiaofan
d842711a8a
Setup config file for embedded etcd (#22076)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-09 22:04:33 +08:00
jaime
68754f978b
Add a segment metric for count number of binlog files (#22085)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-09 19:24:32 +08:00
Jiquan Long
883a79e4b8
Change default max shards num to 64 (#22016) (#22017)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-02-07 14:43:57 +08:00
jaime
a10620c5e8
Modify default value of some configurations (#22012)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-07 10:05:56 +08:00
jaime
f16bbe711d
Add a segment seal policy by number of binlog files (#21941)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-06 14:55:54 +08:00
cai.zhang
85dd8912eb
Reduce scheduler duration of index task from 1s to 500ms (#21893)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-02-02 15:01:51 +08:00
wei liu
926c88ff18
resource group impl (#21895)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-01 19:03:51 +08:00
Xiaofan
709bf35b9a
Remove frequent bulkload log (#21822)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-20 00:27:45 +08:00
Enwei Jiao
cae6d48f36
Disable time-travel by default (#21788)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-18 16:03:44 +08:00
SimFG
1ac30c7bd8
Make the rbac error message clearer (#21646)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-01-12 13:55:42 +08:00
SimFG
4bcfad5e0d
Add the superuser config (#21611)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-01-10 20:25:40 +08:00
SimFG
080bed9083
Improvement apis and error messages about the graceful stop (#21580)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-01-09 16:53:37 +08:00
SimFG
bdced9c3ca
Support the graceful stop for the indexnode (#21556)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-01-06 10:59:35 +08:00
SimFG
3d0359f9a5
Support the graceful stop for the query node (#21528)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-01-05 20:07:36 +08:00
congqixia
fbeec8c487
Add guarantee and serviceable lag too large check (#21441) (#21467)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-05 16:53:36 +08:00
wei liu
dc94fb61ac
fix auth config unmarshal (#21474)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-03 10:13:34 +08:00
xige-16
38f9ea5122
Fix invalid single compaction (#21328)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-12-22 16:05:28 +08:00
Bingyi Sun
642d63c57d
update target (#19296) (#21202)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: wei liu <wei.liu@zilliz.com>
2022-12-16 15:23:25 +08:00
Enwei Jiao
184d3c3583
Make TopK Limit configurable (#21155)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-12 23:01:24 +08:00
Xiaofan
f02ad7b71a
Fix flush didn't respect binaryvector and other schemas (#21119)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-12 10:33:26 +08:00
Ten Thousand Leaves
6dee58685e
Increase MaxWatchDuration and make it configurable (#21010)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-12-07 10:19:19 +08:00
Ten Thousand Leaves
3e62688eb6
Split small segment proportion and small segment compactable proportion (#21012)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-12-07 10:15:19 +08:00
Xiaofan
b8be79fe27
Limit the rpc to 64M (#20844)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-07 10:09:19 +08:00
Enwei Jiao
802f512161
Etcd config source support TLS (#20874) (#20910)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-01 14:19:16 +08:00
yah01
c5f215da67
Refactor task scheduler and executor (#20828) (#20900)
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-30 16:37:17 +08:00
Xiaofan
61baf7dada
Remove some logs from debug to info (#20699)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-29 21:09:15 +08:00
wei liu
5e63b84ff1
support pulsar tenant and auth (#19727) (#20762)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-25 17:11:12 +08:00
wayblink
b494b564e3
Update datanode.segment.insertBufSize default value to 16777216 (#20714)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-11-18 18:35:10 +08:00
bigsheeper
dbcbe4d8f9
Disable tt delay protection (#20658)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-17 20:21:14 +08:00
bigsheeper
6c651d25a4
Ensure compatibility of channel seek position and move syncPeriod to config (#20504) (#20597)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-15 19:39:18 +08:00
MrPresent-Han
f857360cb4
enable delbuf auto flush function to avoid OOM when del msg accumulating (#20213) (#20512)
issue: #19713

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2022-11-14 10:13:06 +08:00
xige-16
2284378f87
Fix spell PQ error (#20527)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-11-12 21:19:04 +08:00
yah01
2e78872be8
Refresh targets if haven't make progress for 5mins (#20382)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-08 19:05:03 +08:00
Xiaofan
bb4777cf6e
Add GCTuner (#20351)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-05 12:07:00 +08:00
zhagnlu
1dac462cae
Change default thread core coefficient (#20120) (#20328) (#20333)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-04 17:49:35 +08:00
zhagnlu
cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 14:39:40 +08:00
shaoyue
9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164)
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-11-01 11:07:35 +08:00