Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create S3 bucket for artifacts to replace the current gha-artifacts bucket #34

Closed
zxiiro opened this issue Feb 20, 2024 · 3 comments
Closed

Create S3 bucket for artifacts to replace the current gha-artifacts bucket #34

zxiiro opened this issue Feb 20, 2024 · 3 comments
Assignees
@zxiiro

Comments

@zxiiro
Copy link
Collaborator

zxiiro commented Feb 20, 2024

The current gha-artifacts S3 bucket is not managed by the PyTorch Foundation AWS account. This issue is to track creation of a solution for this.

The bucket needs to be publicly readable so that developers can fetch the data stored in the bucket but write access also needs to be provided to infrastructure to deploy the artifacts to during build time. Authenticate using OIDC.
@zxiiro zxiiro self-assigned this Feb 20, 2024
@zxiiro
Copy link
Collaborator Author

zxiiro commented Feb 20, 2024

@jeanschmidt does the description sound correct to you?

@zxiiro zxiiro mentioned this issue Feb 20, 2024
Merged
@malfet
Copy link

malfet commented Feb 20, 2024

While we are in this transitory process, would someone mind posting a link, stating that this would not incur egress costs(nor throttling0 while accessing this new bucket from the current AWS account

zxiiro added a commit that referenced this issue Feb 20, 2024
@zxiiro
Create pytorch-ci-artifacts s3 bucket
b95e7dd 
Create the pytorch-ci-artifacts s3 bucket for storing build artifacts
and apply the necessary access policy to the build nodes for the
actions runners. This creates a publicly available S3 Bucket for read
access to anonymous users. While requiring OIDC authentication for
write access by GitHub Actions.

Issue: #34
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
@zxiiro
Copy link
Collaborator Author

zxiiro commented Feb 20, 2024

While we are in this transitory process, would someone mind posting a link, stating that this would not incur egress costs(nor throttling0 while accessing this new bucket from the current AWS account

By default AWS S3 Bucket costs are assigned to the owner of the bucket. There is a configuration that can be enabled however to assign the costs to the requestor to pay for the transfer costs.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html?icmpid=docs_amazons3_console

@zxiiro zxiiro closed this as completed Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zxiiro zxiiro

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zxiiro @malfet