From 96ba559444d05cdcb62dfa8d454bf20a92b32a78 Mon Sep 17 00:00:00 2001 From: Dengke Tang Date: Thu, 7 Dec 2023 10:24:57 -0800 Subject: [PATCH] don't format --- docs/memory_aware_request_execution.md | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/docs/memory_aware_request_execution.md b/docs/memory_aware_request_execution.md index b456477af..2b104ce44 100644 --- a/docs/memory_aware_request_execution.md +++ b/docs/memory_aware_request_execution.md @@ -11,12 +11,11 @@ will go into more detail on aspects of those changes and how the affect the client. ### Memory Reuse - At the basic level, CRT S3 client starts with a meta request for operation like put or get, breaks it into smaller part-sized requests and executes those in parallel. CRT S3 client used to allocate part sized buffer for each of those -requests and release it right after the request was done. That approach, -resulted in a lot of very short lived allocations and allocator thrashing, +requests and release it right after the request was done. That approach, +resulted in a lot of very short lived allocations and allocator thrashing, overall leading to memory use spikes considerably higher than whats needed. To address that, the client is switching to a pooled buffer approach, discussed below. @@ -25,16 +24,15 @@ Note: approach described below is work in progress and concentrates on improving the common cases (default 8mb part sizes and part sizes smaller than 64mb). Several observations about the client usage of buffers: - -* Client does not automatically switch to buffers above default 8mb for upload, until - upload passes 10, 000 parts (~80 GB). -* Get operations always use either the configured part size or default of 8mb. - Part size for get is not adjusted, since there is no 10, 000 part limitation. -* Both Put and Get operations go through fill and drain phases. Ex. for Put, the +- Client does not automatically switch to buffers above default 8mb for upload, until + upload passes 10,000 parts (~80 GB). +- Get operations always use either the configured part size or default of 8mb. + Part size for get is not adjusted, since there is no 10,000 part limitation. +- Both Put and Get operations go through fill and drain phases. Ex. for Put, the client first schedules a number of reads to 'fill' the buffers from the source and as those reads complete, the buffer are send over to the networking layer are 'drained' -* individual uploadParts or ranged gets operations typically have a similar +- individual uploadParts or ranged gets operations typically have a similar lifespan (with some caveats). in practice part buffers are acquired/released in bulk at the same time @@ -50,11 +48,10 @@ Primary memory area is split into blocks of fixed size (part size if defined or subdivided into part sized chunks. Pool allocates and releases in chunk sizes only, and supports acquiring several chunks (up to 4) at once. -Blocks are kept around while there are ongoing requests and are released async, +Blocks are kept around while there are ongoing requests and are released async, when there is low pressure on memory. ### Scheduling - Running out of memory is a terminal condition within CRT and in general its not practical to try to set overall memory limit on all allocations, since it dramatically increases the complexity of the code that deals with cases where @@ -64,7 +61,7 @@ Comparatively, majority of memory usage within S3 Client comes from buffers allocated for Put/Get parts. So to control memory usage, the client will concentrate on controlling the number of buffers allocated. Effectively, this boils down to a back pressure mechanism of limiting the number of parts -scheduled as memory gets closer to the limit. Memory used for other resources, +scheduled as memory gets closer to the limit. Memory used for other resources, ex. http connections data, various supporting structures, are not actively controlled and instead some memory is taken out from overall limit.