Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(report): Add week 11 CI Scanner report #273

Merged
merged 1 commit into from
Sep 2, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions docs/2024/ci-scanner/updates/2024-08-15.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: Week 11
author: Rajul Jha
tags: [gsoc24, CI]
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0

SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
-->

# Week 11
*(Aug, 09 2024 - Aug 15, 2024)*

## Meeting 1
*(Aug 14, 2024)*

## Attendees

* [Rajul Jha](https://github.com/rajuljha)
* [Gaurav](https://github.com/GMishx)
* [Avinal Kumar](https://github.com/avinal)
* Katharina


## Discussions

* Showed the demo of the download dependencies feature to [Gaurav](https://github.com/GMishx) and got feedback for the same.
* Discussed using the `ununpack` agent for unpacking the downloaded packages rather than the current dependency on python libraries.
* Rather than taking an input directory as input to the `--sbom-path`, it would be better to take a single file as input.
* Software projects only have one `sbom` as a convention. So, if the project is using multiple languages or dependencies, then they are combined to a single sbom file, which can be passed to our scanner.
* We want to implement different parsers for different languages. So, we need a parent class of the parsers whose job is to set which parser should handle a particular component.
* We can differentiate between languages and which parser to use based on the `purl` for the component.
* Got a new feature request in order to allow scanning a particular directory by providing the path to it via CLI.


## Work Done

* Implemented the feature of scanning dependencies using `fossologyscanner`.
* Introduced a module `ScanDeps` that has a Parsers file that contains specific parsers for each supported language.
* Currently, it has `PythonParser` only. Working on `NPMParser`. The job of the parsers is to take the sbom file and extract the download urls from it,
save it in a particular format like a tuple of `(<package_name> , <download_url>)`.
* These results are then fed to a `Downloader` class that has methods to concurrently download these packages from the urls, and then extract them
from their respective formats to a directory called `pkg_downloads`.
* The `download_concurrently` method improves the speed of downloading and exporting packages significantly by allowing multiple downloads to run on different threads parallelly.
* Started working on the new feature and improving the different parsers.


## Planning for next week

* Complete the PythonParser and raise a PR for the same.
* Complete the directory scan feature and raise a PR.
* Start working on `NPMParser` next.