mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-07 01:28:27 +08:00
[skip ci]Format markdown doc for milvus_drop_collection_en.md (#10521)
Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>
This commit is contained in:
parent
2ee7463704
commit
f5bd5d8f4b
@ -1,4 +1,5 @@
|
|||||||
# Drop Collection
|
# Drop Collection
|
||||||
|
|
||||||
`Milvus 2.0` uses `Collection` to represent a set of data, like `Table` in traditional database. Users can create or drop `Collection`. Altering the `Schema` of `Collection` is not supported yet. This article introduces the execution path of `Drop Collection`. At the end of this article, you should know which components are involved in `Drop Collection`.
|
`Milvus 2.0` uses `Collection` to represent a set of data, like `Table` in traditional database. Users can create or drop `Collection`. Altering the `Schema` of `Collection` is not supported yet. This article introduces the execution path of `Drop Collection`. At the end of this article, you should know which components are involved in `Drop Collection`.
|
||||||
|
|
||||||
The execution flow of `Drop Collection` is shown in the following figure:
|
The execution flow of `Drop Collection` is shown in the following figure:
|
||||||
@ -6,6 +7,7 @@ The execution flow of `Drop Collection` is shown in the following figure:
|
|||||||

|

|
||||||
|
|
||||||
1. Firstly, `SDK` sends a `DropCollection` request to `Proxy` via `Grpc`, the `proto` is defined as follows:
|
1. Firstly, `SDK` sends a `DropCollection` request to `Proxy` via `Grpc`, the `proto` is defined as follows:
|
||||||
|
|
||||||
```proto
|
```proto
|
||||||
service MilvusService {
|
service MilvusService {
|
||||||
...
|
...
|
||||||
@ -23,6 +25,7 @@ message DropCollectionRequest {
|
|||||||
```
|
```
|
||||||
|
|
||||||
2. Once the `DropCollection` request is received, the `Proxy` would wrap this request into `DropCollectionTask`, and push this task into `DdTaskQueue` queue. After that, `Proxy` would call `WatiToFinish` method to wait until the task is finished.
|
2. Once the `DropCollection` request is received, the `Proxy` would wrap this request into `DropCollectionTask`, and push this task into `DdTaskQueue` queue. After that, `Proxy` would call `WatiToFinish` method to wait until the task is finished.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
type task interface {
|
type task interface {
|
||||||
TraceCtx() context.Context
|
TraceCtx() context.Context
|
||||||
@ -53,20 +56,24 @@ type DropCollectionTask struct {
|
|||||||
```
|
```
|
||||||
|
|
||||||
3. There is a background service in `Proxy`, this service would get the `DropCollectionTask` from `DdTaskQueue`, and execute it in three phases:
|
3. There is a background service in `Proxy`, this service would get the `DropCollectionTask` from `DdTaskQueue`, and execute it in three phases:
|
||||||
- `PreExecute`, do some static checking at this phase, such as check if `Collection Name` is legal etc.
|
|
||||||
- `Execute`, at this phase, `Proxy` would send `DropCollection` request to `RootCoord` via `Grpc`, and wait the response, the `proto` is defined as below:
|
|
||||||
```proto
|
|
||||||
service RootCoord {
|
|
||||||
...
|
|
||||||
|
|
||||||
rpc DropCollection(milvus.DropCollectionRequest) returns (common.Status) {}
|
- `PreExecute`, do some static checking at this phase, such as check if `Collection Name` is legal etc.
|
||||||
|
- `Execute`, at this phase, `Proxy` would send `DropCollection` request to `RootCoord` via `Grpc`, and wait the response, the `proto` is defined as below:
|
||||||
|
|
||||||
...
|
```proto
|
||||||
}
|
service RootCoord {
|
||||||
```
|
...
|
||||||
- `PostExecute`, `Proxy` would delete `Collection`'s meta from global meta table at this phase.
|
|
||||||
|
rpc DropCollection(milvus.DropCollectionRequest) returns (common.Status) {}
|
||||||
|
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- `PostExecute`, `Proxy` would delete `Collection`'s meta from global meta table at this phase.
|
||||||
|
|
||||||
4. `RootCoord` would wrap the `DropCollection` request into `DropCollectionReqTask`, and then call function `executeTask`. `executeTask` would return until the `context` is done or `DropCollectionReqTask.Execute` is returned.
|
4. `RootCoord` would wrap the `DropCollection` request into `DropCollectionReqTask`, and then call function `executeTask`. `executeTask` would return until the `context` is done or `DropCollectionReqTask.Execute` is returned.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
type reqTask interface {
|
type reqTask interface {
|
||||||
Ctx() context.Context
|
Ctx() context.Context
|
||||||
@ -88,6 +95,7 @@ type DropCollectionReqTask struct {
|
|||||||
7. `RootCoord` would alloc a timestamp from `TSO` before deleting `Collection`'s meta from `metaTable`. This timestamp is considered as the point when the collection was deleted.
|
7. `RootCoord` would alloc a timestamp from `TSO` before deleting `Collection`'s meta from `metaTable`. This timestamp is considered as the point when the collection was deleted.
|
||||||
|
|
||||||
8. `RootCoord` would send a message of `DropCollectionRequest` into `MsgStream`. Thus other components, who have subscribed to the `MsgStream`, would be notified. The `Proto` of `DropCollectionRequest` is defined as below:
|
8. `RootCoord` would send a message of `DropCollectionRequest` into `MsgStream`. Thus other components, who have subscribed to the `MsgStream`, would be notified. The `Proto` of `DropCollectionRequest` is defined as below:
|
||||||
|
|
||||||
```proto
|
```proto
|
||||||
message DropCollectionRequest {
|
message DropCollectionRequest {
|
||||||
common.MsgBase base = 1;
|
common.MsgBase base = 1;
|
||||||
@ -102,6 +110,7 @@ message DropCollectionRequest {
|
|||||||
9. After these operations, `RootCoord` would update internal timestamp.
|
9. After these operations, `RootCoord` would update internal timestamp.
|
||||||
|
|
||||||
10. Then `RootCoord` would start a `ReleaseCollection` request to `QueryCoord` via `Grpc` , notify `QueryCoord` to release all resources that related to this `Collection`. This `Grpc` request is done in another `goroutine`, so it would not block the main thread. The `proto` is defined as follows:
|
10. Then `RootCoord` would start a `ReleaseCollection` request to `QueryCoord` via `Grpc` , notify `QueryCoord` to release all resources that related to this `Collection`. This `Grpc` request is done in another `goroutine`, so it would not block the main thread. The `proto` is defined as follows:
|
||||||
|
|
||||||
```proto
|
```proto
|
||||||
service QueryCoord {
|
service QueryCoord {
|
||||||
...
|
...
|
||||||
@ -120,6 +129,7 @@ message ReleaseCollectionRequest {
|
|||||||
```
|
```
|
||||||
|
|
||||||
11. At last, `RootCoord` would send `InvalidateCollectionMetaCache` request to each `Proxy`, notify `Proxy` to remove `Collection`'s meta. The `proto` is defined as follows:
|
11. At last, `RootCoord` would send `InvalidateCollectionMetaCache` request to each `Proxy`, notify `Proxy` to remove `Collection`'s meta. The `proto` is defined as follows:
|
||||||
|
|
||||||
```proto
|
```proto
|
||||||
service Proxy {
|
service Proxy {
|
||||||
...
|
...
|
||||||
@ -143,33 +153,39 @@ message InvalidateCollMetaCacheRequest {
|
|||||||
13. `QueryCoord` would wrap `ReleaseCollection` into `ReleaseCollectionTask`, and push the task into `TaskScheduler`
|
13. `QueryCoord` would wrap `ReleaseCollection` into `ReleaseCollectionTask`, and push the task into `TaskScheduler`
|
||||||
|
|
||||||
14. There is a background service in `QueryCoord`. This service would get the `ReleaseCollectionTask` from `TaskScheduler`, and execute it in three phases:
|
14. There is a background service in `QueryCoord`. This service would get the `ReleaseCollectionTask` from `TaskScheduler`, and execute it in three phases:
|
||||||
|
|
||||||
- `PreExecute`, `ReleaseCollectionTask` would only print debug log at this phase.
|
- `PreExecute`, `ReleaseCollectionTask` would only print debug log at this phase.
|
||||||
- `Execute`, there are two jobs at this phase:
|
- `Execute`, there are two jobs at this phase:
|
||||||
- send a `ReleaseDQLMessageStream` request to `RootCoord` via `Grpc`, `RootCoord` would redirect the `ReleaseDQLMessageStream` request to each `Proxy`, and notify the `Proxy` that stop processing any message of this `Collection` anymore. The `proto` is defined as follows:
|
|
||||||
```proto
|
|
||||||
message ReleaseDQLMessageStreamRequest {
|
|
||||||
common.MsgBase base = 1;
|
|
||||||
int64 dbID = 2;
|
|
||||||
int64 collectionID = 3;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
- send a `ReleaseCollection` request to each `QueryNode` via `Grpc`, and notify the `QueryNode` to release all the resources related to this `Collection`, including `Index`, `Segment`, `FlowGraph`, etc. `QueryNode` would no longer read any message from this `Collection`'s `MsgStream` anymore
|
|
||||||
```proto
|
|
||||||
service QueryNode {
|
|
||||||
...
|
|
||||||
|
|
||||||
rpc ReleaseCollection(ReleaseCollectionRequest) returns (common.Status) {}
|
- send a `ReleaseDQLMessageStream` request to `RootCoord` via `Grpc`, `RootCoord` would redirect the `ReleaseDQLMessageStream` request to each `Proxy`, and notify the `Proxy` that stop processing any message of this `Collection` anymore. The `proto` is defined as follows:
|
||||||
|
|
||||||
...
|
```proto
|
||||||
}
|
message ReleaseDQLMessageStreamRequest {
|
||||||
|
common.MsgBase base = 1;
|
||||||
|
int64 dbID = 2;
|
||||||
|
int64 collectionID = 3;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- send a `ReleaseCollection` request to each `QueryNode` via `Grpc`, and notify the `QueryNode` to release all the resources related to this `Collection`, including `Index`, `Segment`, `FlowGraph`, etc. `QueryNode` would no longer read any message from this `Collection`'s `MsgStream` anymore
|
||||||
|
|
||||||
|
```proto
|
||||||
|
service QueryNode {
|
||||||
|
...
|
||||||
|
|
||||||
|
rpc ReleaseCollection(ReleaseCollectionRequest) returns (common.Status) {}
|
||||||
|
|
||||||
|
...
|
||||||
|
}
|
||||||
|
|
||||||
|
message ReleaseCollectionRequest {
|
||||||
|
common.MsgBase base = 1;
|
||||||
|
int64 dbID = 2;
|
||||||
|
int64 collectionID = 3;
|
||||||
|
int64 nodeID = 4;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
message ReleaseCollectionRequest {
|
|
||||||
common.MsgBase base = 1;
|
|
||||||
int64 dbID = 2;
|
|
||||||
int64 collectionID = 3;
|
|
||||||
int64 nodeID = 4;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
- `PostExecute`, `ReleaseCollectionTask` would only print debug log at this phase.
|
- `PostExecute`, `ReleaseCollectionTask` would only print debug log at this phase.
|
||||||
|
|
||||||
15. After these operations, `QueryCoord` would send `ReleaseCollection`'s response to `RootCoord`.
|
15. After these operations, `QueryCoord` would send `ReleaseCollection`'s response to `RootCoord`.
|
||||||
@ -180,6 +196,7 @@ message InvalidateCollMetaCacheRequest {
|
|||||||
|
|
||||||
17. In `DataNode`, each `MsgStream` will have a `FlowGraph`, which processes all messages. When the `DataNode` receives the message of `DropCollectionRequest`, `DataNode` would notify `BackGroundGC`, which is a background service on `DataNode`, to release resources.
|
17. In `DataNode`, each `MsgStream` will have a `FlowGraph`, which processes all messages. When the `DataNode` receives the message of `DropCollectionRequest`, `DataNode` would notify `BackGroundGC`, which is a background service on `DataNode`, to release resources.
|
||||||
|
|
||||||
*Notes*:
|
_Notes_:
|
||||||
|
|
||||||
1. Currently, the `DataCoord` doesn't have response to the `DropCollection`. So the `Collection`'s `segment meta` still exists in the `DataCoord`'s `metaTable`, and the `Binlog` files belonging to this `Collection` still exist in the persistent storage.
|
1. Currently, the `DataCoord` doesn't have response to the `DropCollection`. So the `Collection`'s `segment meta` still exists in the `DataCoord`'s `metaTable`, and the `Binlog` files belonging to this `Collection` still exist in the persistent storage.
|
||||||
2. Currently, the `IndexCoord` doesn't have response to the `DropCollection`. So the `Collection`'s `index file` still exists in the persistent storage.
|
2. Currently, the `IndexCoord` doesn't have response to the `DropCollection`. So the `Collection`'s `index file` still exists in the persistent storage.
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user