Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/sync deleted repos #423

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
- [File renamed/moves/deleted](#file-renamedmovesdeleted)
- [node\_modules, tests \& fixtures](#node_modules-tests--fixtures)
- [Detecting \& importing new files not already monitored in Snyk](#detecting--importing-new-files-not-already-monitored-in-snyk)
- [Repository is archived](#repository-is-archived)
- [Repository is archived or deleted](#repository-is-archived-or-deleted)
- [Kick off sync](#kick-off-sync)
- [1. Set the env vars](#1-set-the-env-vars)
- [2. Download \& run](#2-download--run)
Expand Down Expand Up @@ -77,9 +77,9 @@ Any projects that were imported but match the default exclusions list (deemed to
While analyzing each target known to Snyk any new Snyk supported files found in the repo that do not have a corresponding project in Snyk will be imported in batches. Any files matching the default or user provided `exclusionGlobs` will be ignored.
If a file has a corresponding de-activated project in Snyk, it will not be brought in again. Activate manually or via API if it should be active.

## Repository is archived
## Repository is archived or deleted

If the repository is now marked as archived, all relevant Snyk projects will be de-activated.
If the repository is now marked as archived or has been deleted (API returns 404), all relevant Snyk projects will be de-activated.
# Kick off sync

`sync` command will analyze existing projects & targets (repos) in Snyk organization and determine if any changes are needed.
Expand Down
51 changes: 39 additions & 12 deletions src/scripts/sync/sync-projects-per-target.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ export async function syncProjectsForTarget(

debug(`Syncing projects for target ${target.attributes.displayName}`);
let targetMeta: RepoMetaData;
let isDeleted = false;
const origin = target.attributes
.origin as SupportedIntegrationTypesUpdateProject;
const targetData = getTargetConverter(origin)(target);
Expand All @@ -68,19 +69,45 @@ export async function syncProjectsForTarget(
targetMeta = await getMetaDataGenerator(origin)(targetData, host);
} catch (e) {
//TODO: if repo is deleted, deactivate all projects
debug(e);
const error = `Getting default branch via ${origin} API failed with error: ${e.message}`;
projects.map((project) => {
failed.add({
errorMessage: error,
projectPublicId: project.id,
type: ProjectUpdateType.BRANCH,
from: project.branch!,
to: targetMeta.branch,
dryRun: config.dryRun,
target,
debug(`Failed to get metadata ${JSON.stringify(targetData)}: ` + e);
if (e.status === 404) {
// TODO: when else could you get a 404? Manually check
isDeleted = true;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how to make this safer? check the token can see org info?

} else {
const error = `Getting metadata from ${origin} API failed with error: ${e.message}`;
projects.map((project) => {
failed.add({
errorMessage: error,
projectPublicId: project.id,
type: ProjectUpdateType.BRANCH,
from: project.branch!,
to: 'unknown',
dryRun: config.dryRun,
target,
});
});
});
return {
updated: Array.from(updated).map((t) => ({ ...t, target })),
failed: Array.from(failed).map((t) => ({ ...t, target })),
};
}
}

if (isDeleted || targetMeta!.archived) {
const res = await bulkDeactivateProjects(
requestManager,
orgId,
projects,
config.dryRun,
);

res.updated.map((t) => ({ ...t, target })).forEach((i) => updated.add(i));
res.failed.map((t) => ({ ...t, target })).forEach((i) => failed.add(i));

return {
updated: Array.from(updated).map((t) => ({ ...t, target })),
failed: Array.from(failed).map((t) => ({ ...t, target })),
};
}

const deactivate = [];
Expand Down
20 changes: 10 additions & 10 deletions test/scripts/sync/clone-and-analyze.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ describe('cloneAndAnalyze', () => {
];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: 'https://github.com/snyk-fixtures/monorepo-simple.git',
sshUrl: 'git@github.com:snyk-fixtures/monorepo-simple.git',
};
Expand Down Expand Up @@ -130,8 +130,8 @@ describe('cloneAndAnalyze', () => {
];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: 'https://github.com/snyk-fixtures/monorepo-simple.git',
sshUrl: 'git@github.com:snyk-fixtures/monorepo-simple.git',
};
Expand Down Expand Up @@ -205,8 +205,8 @@ describe('cloneAndAnalyze', () => {
];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: 'https://github.com/snyk-fixtures/monorepo-simple.git',
sshUrl: 'git@github.com:snyk-fixtures/monorepo-simple.git',
};
Expand Down Expand Up @@ -298,8 +298,8 @@ describe('cloneAndAnalyze', () => {
];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: 'https://github.com/snyk-fixtures/monorepo-simple.git',
sshUrl: 'git@github.com:snyk-fixtures/monorepo-simple.git',
};
Expand All @@ -324,8 +324,8 @@ describe('cloneAndAnalyze', () => {
const projects: SnykProject[] = [];

const repoMeta: RepoMetaData = {
branch: 'main',
archived: false,
branch: 'main',
cloneUrl: 'https://github.com/snyk-fixtures/no-supported-manifests.git',
sshUrl: 'git@github.com:snyk-fixtures/no-supported-manifests.git',
};
Expand All @@ -350,8 +350,8 @@ describe('cloneAndAnalyze', () => {
const projects: SnykProject[] = [];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: 'https://github.com/snyk-fixtures/empty-repo.git',
sshUrl: 'git@github.com:snyk-fixtures/empty-repo.git',
};
Expand All @@ -376,8 +376,8 @@ describe('cloneAndAnalyze', () => {
const projects: SnykProject[] = [];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl:
'https://github.com/snyk-fixtures/python-requirements-custom-name-inside-folder.git',
sshUrl:
Expand Down Expand Up @@ -414,8 +414,8 @@ describe('cloneAndAnalyze', () => {
const projects: SnykProject[] = [];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: `https://${GHE_URL.host}/snyk-fixtures/mono-repo.git`,
sshUrl: `git@${GHE_URL.host}/snyk-fixtures/mono-repo.git`,
};
Expand Down Expand Up @@ -451,8 +451,8 @@ describe('cloneAndAnalyze', () => {
const projects: SnykProject[] = [];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: `https://${GHE_URL.host}/snyk-fixtures/docker-goof.git`,
sshUrl: `git@${GHE_URL.host}/snyk-fixtures/docker-goof.git`,
};
Expand Down Expand Up @@ -481,8 +481,8 @@ describe('cloneAndAnalyze', () => {
const projects: SnykProject[] = [];

const repoMeta: RepoMetaData = {
branch: 'master',
archived: false,
branch: 'master',
cloneUrl: `https://${GHE_URL.host}/snyk-fixtures/docker-goof.git`,
sshUrl: `git@${GHE_URL.host}/snyk-fixtures/docker-goof.git`,
};
Expand Down
Loading