Skip to content

Commit

Permalink
don't format
Browse files Browse the repository at this point in the history
  • Loading branch information
TingDaoK committed Dec 7, 2023
1 parent 3c92688 commit 96ba559
Showing 1 changed file with 10 additions and 13 deletions.
23 changes: 10 additions & 13 deletions docs/memory_aware_request_execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,11 @@ will go into more detail on aspects of those changes and how the affect the
client.

### Memory Reuse

At the basic level, CRT S3 client starts with a meta request for operation like
put or get, breaks it into smaller part-sized requests and executes those in
parallel. CRT S3 client used to allocate part sized buffer for each of those
requests and release it right after the request was done. That approach,
resulted in a lot of very short lived allocations and allocator thrashing,
requests and release it right after the request was done. That approach,
resulted in a lot of very short lived allocations and allocator thrashing,
overall leading to memory use spikes considerably higher than whats needed. To
address that, the client is switching to a pooled buffer approach, discussed
below.
Expand All @@ -25,16 +24,15 @@ Note: approach described below is work in progress and concentrates on improving
the common cases (default 8mb part sizes and part sizes smaller than 64mb).

Several observations about the client usage of buffers:

* Client does not automatically switch to buffers above default 8mb for upload, until
upload passes 10, 000 parts (~80 GB).
* Get operations always use either the configured part size or default of 8mb.
Part size for get is not adjusted, since there is no 10, 000 part limitation.
* Both Put and Get operations go through fill and drain phases. Ex. for Put, the
- Client does not automatically switch to buffers above default 8mb for upload, until
upload passes 10,000 parts (~80 GB).
- Get operations always use either the configured part size or default of 8mb.
Part size for get is not adjusted, since there is no 10,000 part limitation.
- Both Put and Get operations go through fill and drain phases. Ex. for Put, the
client first schedules a number of reads to 'fill' the buffers from the source
and as those reads complete, the buffer are send over to the networking layer
are 'drained'
* individual uploadParts or ranged gets operations typically have a similar
- individual uploadParts or ranged gets operations typically have a similar
lifespan (with some caveats). in practice part buffers are acquired/released
in bulk at the same time

Expand All @@ -50,11 +48,10 @@ Primary memory area is split into blocks of fixed size (part size if defined or
subdivided into part sized chunks. Pool allocates and releases in chunk sizes
only, and supports acquiring several chunks (up to 4) at once.

Blocks are kept around while there are ongoing requests and are released async,
Blocks are kept around while there are ongoing requests and are released async,
when there is low pressure on memory.

### Scheduling

Running out of memory is a terminal condition within CRT and in general its not
practical to try to set overall memory limit on all allocations, since it
dramatically increases the complexity of the code that deals with cases where
Expand All @@ -64,7 +61,7 @@ Comparatively, majority of memory usage within S3 Client comes from buffers
allocated for Put/Get parts. So to control memory usage, the client will
concentrate on controlling the number of buffers allocated. Effectively, this
boils down to a back pressure mechanism of limiting the number of parts
scheduled as memory gets closer to the limit. Memory used for other resources,
scheduled as memory gets closer to the limit. Memory used for other resources,
ex. http connections data, various supporting structures, are not actively
controlled and instead some memory is taken out from overall limit.

Expand Down

0 comments on commit 96ba559

Please sign in to comment.