1211 Commits

Author SHA1 Message Date
congqixia
938c09679c
Fix LRU cache may panic on double close (#20854)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-29 09:47:14 +08:00
Ten Thousand Leaves
4f9a294b2e
Refine segment stat metrics on QueryNode (#20763)
issue: #20760

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-11-28 14:53:14 +08:00
smellthemoon
d9185485f1
Remove TraceID in mq (#20697)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-25 23:11:12 +08:00
yah01
f76ea292d2
Change LRU to a generic type (#20744)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-22 12:11:11 +08:00
xige-16
1ad702cc40
Reduce the memory used by load disk index (#20475)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-11-21 19:35:11 +08:00
wayblink
b32f99e2e4
Update datanode.segment.insertBufSize default value to 16777216 (#20713)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-11-21 19:19:11 +08:00
jaime
d84cda22ad
Fix start fail with kafka model (#20726)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-21 17:49:12 +08:00
xige-16
34c88cea32
Fix GetDiskUsedSize not work (#20474)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-11-21 17:47:17 +08:00
xige-16
82570e057c
Limit MinDim to build disk index (#20724)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-11-21 17:43:10 +08:00
xige-16
81b6703201
Fix spell PQ error (#20725)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-11-21 16:35:11 +08:00
smellthemoon
f5ab719f21
timestamp decided if the pks were the same (#20166)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-21 10:55:10 +08:00
groot
bf2107ecf5
Fix a regression of local storage (#20653)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-21 10:19:10 +08:00
jaime
0a5270e8c0
Fix alter collection hang (#20694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-21 10:09:10 +08:00
SimFG
dffcd974ff
Make sure that the segment path matches the segment id (#20677)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-11-18 15:35:09 +08:00
Xiaofan
633a749880
Recude IndexCodec Load Memory (#20621)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-18 10:47:08 +08:00
bigsheeper
fef87f879c
Disable tt delay protection (#20657)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-17 20:21:14 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
Enwei Jiao
19524a5344
Fix nodeID mismatch at standalone mode (#20648)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 17:15:08 +08:00
groot
8da44638ac
Refine json parser for import (#20545)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-16 19:05:08 +08:00
Xiaofan
58ae746f69
Fix GC Tuner Overflow (#20497)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-15 10:43:06 +08:00
Xiaofan
af66a0b621
Fix Time Tick session2ChanTs map use the same nodeID (#20537)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-14 14:41:11 +08:00
groot
e217b951ee
Fix bulkinsert bug for int64 value (#20491)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-14 10:15:10 +08:00
bigsheeper
fc15789da9
Ensure compatibility of channel seek position and move syncPeriod to config (#20504)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-12 21:09:04 +08:00
wei liu
701e7b5a6e
support pulsar tenant and auth (#19727)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-11 10:55:05 +08:00
bigsheeper
cd19d99ad7
Add channel level checkpoint (#20350)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-10 22:13:04 +08:00
yah01
663bdf3399
Add RootPath field back (#20488)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-10 19:43:03 +08:00
aoiasd
fca2b71e28
add access log for proxy (#19927)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-11-10 17:09:06 +08:00
groot
53850e340c
Dont expire pending import tasks (#20431)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-10 10:25:08 +08:00
wei liu
f925fa7661
fix check segment stale task (#20401)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-08 19:43:05 +08:00
Enwei Jiao
7e56e7a976
Fix datarace at Setlogger (#20376)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-08 14:33:03 +08:00
wayblink
a7bed1c927
Pass backup flag through Import request options (#20334)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-11-08 11:33:03 +08:00
wei liu
c5cd92d36e
update target (#19296)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-07 19:37:04 +08:00
MrPresent-Han
21b54709a2
enable delbuf auto flush function to avoid OOM when del msg accumulating (#20213)
issue: #19713

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: MrPresent-Han <zilliz@zillizdeMacBook-Pro-2.local>
2022-11-07 18:49:02 +08:00
groot
b847c425e1
Fix listImportTasks bug (#20301)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-07 17:11:02 +08:00
Xiaofan
0d18336b21
Fix GCTuner calculation overflow (#20354)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-07 12:17:05 +08:00
jaime
709a0a94e9
Add some msgstream metrics (#20296)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-07 10:15:02 +08:00
xige-16
4a66965df4
Delete RAW_DATA copy when load IVF_FLAT index data (#20274)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-11-05 17:33:05 +08:00
Xiaofan
ffd52f0d08
Add GC Tuner (#20321)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-04 22:25:01 +08:00
Enwei Jiao
956c5e1b9d
Make Params singleton (#20088)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
zhagnlu
15b2d1cace
Change default thread core coefficient (#20120) (#20328)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-04 12:41:35 +08:00
zhagnlu
cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 14:39:40 +08:00
Ten Thousand Leaves
5708352a80
Improve DataCoord and DataNode logs (#20265)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-11-03 12:15:35 +08:00
groot
cd3b3d3da6
Only allow one file for row-based import (#20234)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-02 10:23:35 +08:00
shaoyue
9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164)
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-11-01 11:07:35 +08:00
bigsheeper
a2d5947fe4
Change limiter burst type to float64 (#20215)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-31 20:59:34 +08:00
groot
c6151ad351
Parse utf32 string of numpy file (#20176)
Signed-off-by: groot <yihua.mo@zilliz.com>

Signed-off-by: groot <yihua.mo@zilliz.com>
2022-10-31 17:39:34 +08:00
Xiaofan
4136009a9a
Support load segments/channels in parallel (#20036)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-31 13:55:33 +08:00
Jiquan Long
8d0cc4226c
Fix IVF_SQ nbits check (#20183)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-10-31 10:13:34 +08:00
bigsheeper
bddd47c60f
Prevent limiter burst overflow (#20153)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-28 19:05:32 +08:00
bigsheeper
b949fa0ea9
Increase default ttProtection max delay (#20125)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-27 17:37:32 +08:00