Elastic #171
Elastic #171
Conversation
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: gitworkflows <118260833+gitworkflows@users.noreply.github.com>
* fix: test_min_password_lenght_400 * Delete integrations/pcap_analyzers/config/suricata/rules/classification.config Signed-off-by: gitworkflows <118260833+gitworkflows@users.noreply.github.com> * Delete integrations/pcap_analyzers/config/suricata/rules/suricata.rules Signed-off-by: gitworkflows <118260833+gitworkflows@users.noreply.github.com> --------- Signed-off-by: gitworkflows <118260833+gitworkflows@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Signed-off-by: KhulnaSoft bot <43526132+khulnasoft-bot@users.noreply.github.com>
Reviewer's Guide by SourceryThis PR introduces significant changes to improve the plugin management system in ThreatMatrix, particularly around Analyzers, Pivots and Playbooks. The main changes include adding new UI forms for creating/editing plugins, refactoring the runtime configuration handling, improving the VT integration, and adding support for new file types. The changes also include various bug fixes and improvements to the codebase. Class diagram for VirusTotalv3BaseMixin and VirusTotalv3AnalyzerMixinclassDiagram
class VirusTotalv3BaseMixin {
+url: str
+url_sub_path: str
+_api_key_name: str
+headers() dict
+config(runtime_configuration: Dict)
+_perform_get_request(uri: str, ignore_404: bool, **kwargs) Dict
+_perform_post_request(uri: str, ignore_404: bool, **kwargs)
+_perform_request(uri: str, method: str, ignore_404: bool, **kwargs) Dict
+_get_relationship_for_classification(obs_clfn: str, iocs: bool) List
+_get_requests_params_and_uri(obs_clfn: str, observable_name: str, iocs: bool) Tuple[Dict, str, List]
+_fetch_behaviour_summary(observable_name: str) Dict
+_fetch_sigma_analyses(observable_name: str) Dict
+_vt_download_file(file_hash: str) bytes
+_vt_intelligence_search(query: str, limit: int, order_by: str) Dict
+_vt_get_iocs_from_file(sample_hash: str) Dict
}
class VirusTotalv3AnalyzerMixin {
+max_tries: int
+poll_distance: int
+rescan_max_tries: int
+rescan_poll_distance: int
+include_behaviour_summary: bool
+include_sigma_analyses: bool
+force_active_scan_if_old: bool
+days_to_say_that_a_scan_is_old: int
+relationships_to_request: list
+relationships_elements: int
+_get_relationship_limit(relationship: str) int
+_vt_get_relationships(observable_name: str, relationships_requested: list, uri: str, result: dict)
+_get_url_prefix_postfix(result: Dict) Tuple[str, str]
+_vt_scan_file(md5: str, rescan_instead: bool) Dict
+_vt_poll_for_report(observable_name: str, params: Dict, uri: str, obs_clfn: str) Dict
+_vt_include_behaviour_summary(result: Dict, observable_name: str) Dict
+_vt_include_sigma_analyses(result: Dict, observable_name: str) Dict
+_vt_get_report(obs_clfn: str, observable_name: str) Dict
}
VirusTotalv3AnalyzerMixin --|> VirusTotalv3BaseMixin
Class diagram for ScanForm componentclassDiagram
class ScanForm {
-searchParams
-observableParam
-isSampleParam
-investigationIdParam
-parentIdParam
-guideState
-setGuideState
-formik
-organizationPluginsState
-visualizersLoading
-pivotsLoading
-analyzersError
-connectorsError
-playbooksError
-playbooks
-selectObservableType(value)
-updateAdvancedConfig(tags, newClassification, oldClassification)
}
ScanForm --> AnalyzersMultiSelectDropdownInput
ScanForm --> ConnectorsMultiSelectDropdownInput
ScanForm --> PlaybookMultiSelectDropdownInput
ScanForm --> TLPSelectInput
ScanForm --> ScanConfigSelectInput
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 14262919 | Triggered | Username Password | 2d9ca9e | tests/auth/test_auth.py | View secret |
| 13180230 | Triggered | Username Password | 85780e0 | tests/auth/test_auth.py | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
|
Important Review skippedMore than 25% of the files skipped due to max files limit. The review is being skipped to prevent a low-quality review. 97 files out of 181 files are above the max files limit of 75. Please upgrade to Pro plan to get higher limits. You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
Here's the code health analysis summary for commits Analysis Summary
|
Signed-off-by: gitworkflows <118260833+gitworkflows@users.noreply.github.com>
Signed-off-by: gitworkflows <118260833+gitworkflows@users.noreply.github.com>
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @gitworkflows - I've reviewed your changes - here's some feedback:
Overall Comments:
- Consider consolidating common plugin configuration patterns into shared utilities to reduce code duplication across different plugin types
- Error handling could be more consistent - suggest standardizing error message formats and detail levels across the codebase
Here's what I looked at during the review
- 🟡 General issues: 3 issues found
- 🟢 Security: all looks good
- 🟡 Testing: 3 issues found
- 🟡 Complexity: 1 issue found
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| raise NotImplementedError() | ||
| logger.info(f"requests done to: {response.request.url} ") | ||
| logger.debug(f"text: {response.text}") | ||
| result = response.json() |
There was a problem hiding this comment.
suggestion (bug_risk): Error handling could be improved to handle JSON parsing failures separately
The error variable is used in the error message before we know if json parsing succeeded. Consider handling the JSON parsing in a separate try-except block.
| result = response.json() | |
| try: | |
| result = response.json() | |
| except ValueError: | |
| logger.error(f"Invalid JSON response from {response.request.url}") | |
| raise |
| ) | ||
| return attrs | ||
|
|
||
| def create(self, validated_data): |
There was a problem hiding this comment.
suggestion: Duplicated plugin config handling logic in create and update methods
Consider extracting the plugin config handling logic into a helper method to avoid code duplication between create and update.
def _handle_plugin_config(self, validated_data):
return validated_data.pop("plugin_config", {})
def create(self, validated_data):
plugin_config = self._handle_plugin_config(validated_data)
pc = super().create(validated_data)| # optional certificate | ||
| verify = True # defualt | ||
| if hasattr(self, "_certificate") and self._certificate: | ||
| self.__cert_file = NamedTemporaryFile(mode="w") |
There was a problem hiding this comment.
issue: Temporary certificate file should be properly managed and cleaned up
Use a context manager (with statement) to ensure the temporary file is properly closed and removed. Consider using the delete=True parameter with NamedTemporaryFile.
| def test_create(self): | ||
| # invalid fields | ||
| response = self.client.post( | ||
| self.URL, | ||
| data={ | ||
| "name": "TestCreate", | ||
| "python_module": "basic_observable_analyzer.BasicObservableAnalyzer", | ||
| }, | ||
| format="json", | ||
| ) |
There was a problem hiding this comment.
suggestion (testing): Consider adding test for invalid plugin configuration
While the test covers basic creation and plugin configuration, it would be valuable to add a test case for invalid plugin configuration to ensure proper error handling.
def test_create(self):
# Test invalid fields
response = self.client.post(
self.URL,
data={
"name": "TestCreate",
"python_module": "basic_observable_analyzer.BasicObservableAnalyzer",
"plugin_config": {"invalid_key": "invalid_value"}
},
format="json",
)
self.assertEqual(response.status_code, 400)| screen.getByText( | ||
| `${pluginType_} with name ${pluginName} deleted with success`, | ||
| ), | ||
| ).toBeInTheDocument(); |
There was a problem hiding this comment.
suggestion (testing): Add test for successful playbook edit
The test only covers the loading state. Consider adding a test case that verifies the successful editing of a playbook configuration.
|
|
||
| from api_app.analyzers_manager.file_analyzers.iocextract import IocExtract | ||
| from api_app.models import Job | ||
| from tests import CustomTestCase |
There was a problem hiding this comment.
issue (code-quality): Don't import test modules. (dont-import-test-modules)
Explanation
Don't import test modules.Tests should be self-contained and don't depend on each other.
If a helper function is used by multiple tests,
define it in a helper module,
instead of importing one test from the other.
|
|
||
| from api_app.analyzers_manager.file_analyzers.lnk_info import LnkInfo | ||
| from api_app.models import Job | ||
| from tests import CustomTestCase |
There was a problem hiding this comment.
issue (code-quality): Don't import test modules. (dont-import-test-modules)
Explanation
Don't import test modules.Tests should be self-contained and don't depend on each other.
If a helper function is used by multiple tests,
define it in a helper module,
instead of importing one test from the other.
|
|
||
| from api_app.analyzers_manager.file_analyzers.onenote import OneNoteInfo | ||
| from api_app.models import Job | ||
| from tests import CustomTestCase |
There was a problem hiding this comment.
issue (code-quality): Don't import test modules. (dont-import-test-modules)
Explanation
Don't import test modules.Tests should be self-contained and don't depend on each other.
If a helper function is used by multiple tests,
define it in a helper module,
instead of importing one test from the other.
|
|
||
| from api_app.analyzers_manager.file_analyzers.pdf_info import PDFInfo | ||
| from api_app.models import Job | ||
| from tests import CustomTestCase |
There was a problem hiding this comment.
issue (code-quality): Don't import test modules. (dont-import-test-modules)
Explanation
Don't import test modules.Tests should be self-contained and don't depend on each other.
If a helper function is used by multiple tests,
define it in a helper module,
instead of importing one test from the other.
|
|
||
| from api_app.analyzers_manager.file_analyzers.strings_info import StringsInfo | ||
| from api_app.models import Job | ||
| from tests import CustomTestCase |
There was a problem hiding this comment.
issue (code-quality): Don't import test modules. (dont-import-test-modules)
Explanation
Don't import test modules.Tests should be self-contained and don't depend on each other.
If a helper function is used by multiple tests,
define it in a helper module,
instead of importing one test from the other.
PR Code Suggestions ✨Explore these optional code suggestions:
|
User description
(Please add to the PR name the issue/s that this PR would close if merged by using a Github keyword. Example:
<feature name>. Closes #999. If your PR is made by a single commit, please add that clause in the commit too. This is all required to automate the closure of related issues.)Description
Please include a summary of the change and link to the related issue.
Type of change
Please delete options that are not relevant.
Checklist
developdumpplugincommand and added it in the project as a data migration. ("How to share a plugin with the community")test_files.zipand you added the default tests for that mimetype in test_classes.py.FREE_TO_USE_ANALYZERSplaybook by following this guide.urlthat contains this information. This is required for Health Checks._monkeypatch()was used in its class to apply the necessary decorators.MockUpResponseof the_monkeypatch()method. This serves us to provide a valid sample for testing.Black,Flake,Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.testsfolder). All the tests (new and old ones) gave 0 errors.DeepSource,Django Doctorsor other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.Important Rules
PR Type
Enhancement, Tests, Configuration changes, Documentation, Formatting
Description
Changes walkthrough 📝
62 files
mixins.py
Introduced VirusTotal API interaction mixins for analyzersapi_app/mixins.py
VirusTotalv3BaseMixinandVirusTotalv3AnalyzerMixinclasses forinteracting with VirusTotal API.
API.
VirusTotal.
virus_total.py
Implemented VirusTotal ingestor for intelligence search and IOCsextractionapi_app/ingestors_manager/ingestors/virus_total.py
VirusTotalclass inheriting fromIngestorandVirusTotalv3BaseMixin.runmethod to perform intelligence search and extract IOCsor download files.
doc_info.py
Enhanced document analysis with URL extraction and macro analysisapi_app/analyzers_manager/file_analyzers/doc_info.py
DocInfoclass to extract URLs and analyze macros.vt3_intelligence_search.py
Refactored VirusTotal intelligence search analyzer with mixinapi_app/analyzers_manager/observable_analyzers/vt/vt3_intelligence_search.py
VirusTotalv3Intelligenceclass to useVirusTotalv3AnalyzerMixin.runmethod to utilize mixin's intelligence search.nvd_cve.py
Implemented NVDDetails analyzer for CVE data retrievalapi_app/analyzers_manager/observable_analyzers/nvd_cve.py
NVDDetailsclass for fetching CVE details from NVD API.serializers.py
Updated serializers for ingestor configuration and reportapi_app/ingestors_manager/serializers.py
IngestorConfigSerializerForMigrationto exclude specificfields.
IngestorReportBISerializerto include additional fields.0121_analyzer_config_lnk_info.py
Add migration for Lnk_Info analyzer configuration.api_app/analyzers_manager/migrations/0121_analyzer_config_lnk_info.py
AnalyzerConfigrelated toLnk_Info.Lnk_Info.basic_observable_analyzer.py
Implement BasicObservableAnalyzer for observable analysis.api_app/analyzers_manager/observable_analyzers/basic_observable_analyzer.py
BasicObservableAnalyzerclass for analyzing observables.certificate verification.
serializers.py
Enhance PivotConfigSerializer with related configurations.api_app/pivots_manager/serializers.py
PivotConfigSerializerto include related analyzer andconnector configurations.
strings_info.py
Enhance StringsInfo analyzer with URL extraction.api_app/analyzers_manager/file_analyzers/strings_info.py
updatemethod.boxjs_scan.py
Enhance BoxJS analyzer with logging and URL extraction.api_app/analyzers_manager/file_analyzers/boxjs_scan.py
BoxJSanalyzer.updatemethod.0123_basic_observable_analyzer.py
Add migration for BasicObservableAnalyzer module.api_app/analyzers_manager/migrations/0123_basic_observable_analyzer.py
BasicObservableAnalyzerPython module.serializers.py
Enhance AnalyzerConfigSerializer with plugin configuration handling.api_app/analyzers_manager/serializers.py
AnalyzerConfigSerializerto handle plugin configurations.views.py
Add rescan action to job viewset.api_app/views.py
rescanaction for job viewset.serializers.py
Enhance PlaybookConfigSerializer with visualizers and editable flag.api_app/playbooks_manager/serializers.py
PlaybookConfigSerializerto include visualizers and make typefield optional.
is_deletabletois_editable.plugin.py
Add PivotConfig type and update PythonConfigSerializer.api_app/serializers/plugin.py
PivotConfigas a new plugin type.parametersfield optional inPythonConfigSerializer.signals.py
Update signal handlers for cache refresh and logging.api_app/signals.py
PythonConfig.LogEntrypost-save signal.views.py
Enhance PivotConfigViewSet with mixins and permissions.api_app/pivots_manager/views.py
PivotConfigViewSetwith additional mixins and permissions.admin.py
Register LogEntry model in admin with restrictions.api_app/admin.py
LogEntrymodel in admin with restricted permissions.LogEntry.artifacts.py
Simplify Artifacts analyzer run method.api_app/analyzers_manager/file_analyzers/artifacts.py
runmethod by removing conditional logic.0051_add_lnk_info_analyzer_free_to_use.py
Add Lnk_Info analyzer to FREE_TO_USE_ANALYZERS.api_app/playbooks_manager/migrations/0051_add_lnk_info_analyzer_free_to_use.py
Lnk_Infoanalyzer inFREE_TO_USE_ANALYZERS.analyzer_extractor.py
Add check for rrdata in Robtex reports.api_app/visualizers_manager/visualizers/passive_dns/analyzer_extractor.py
rrdatakey in Robtex reports.rrdata.compare.py
Simplify Compare pivot logic with get_value method.api_app/pivots_manager/pivots/compare.py
_get_valuemethod withget_valuefrom related reports.should_runmethod logic.lnk_info.py
Add LnkInfo analyzer for LNK file analysis.api_app/analyzers_manager/file_analyzers/lnk_info.py
LnkInfoanalyzer for extracting URIs from LNK files.pylnk3.elastic_templates.py
Add command for Elasticsearch index template management.api_app/management/commands/elastic_templates.py
templates.
update.
pdf_info.py
Enhance PDFInfo analyzer with URI extraction.api_app/analyzers_manager/file_analyzers/pdf_info.py
PDFInfoanalyzer.views.py
Enhance AnalyzerConfigViewSet with mixins and permissions.api_app/analyzers_manager/views.py
AnalyzerConfigViewSetwith additional mixins and permissions.AnalyzerConfig.constants.py
Add HTTPMethods constants and update domain regex.api_app/analyzers_manager/constants.py
HTTPMethodsclass for HTTP method constants.classes.py
Update health_check method with status code handling.api_app/classes.py
health_checkmethod to handle specific HTTP status codes.onenote.py
Add OneNoteInfo analyzer for OneNote file analysis.api_app/analyzers_manager/file_analyzers/onenote.py
OneNoteInfoanalyzer for extracting content from OneNote files.0122_alter_soft_time_limit.py
Alter soft time limit for Droidlysis analyzer.api_app/analyzers_manager/migrations/0122_alter_soft_time_limit.py
Droidlysisanalyzer.views.py
Add debug logging and parent job handling in playbook views.api_app/playbooks_manager/views.py
analyze_multiple_files.app.py
Improve error handling in intercept_box_js_result.integrations/malware_tools_analyzers/app.py
intercept_box_js_result.dumpplugin.py
Add support for reverse descriptors in dumpplugin.api_app/management/commands/dumpplugin.py
_get_objfunction.models.py
Add get_value method to Job model.api_app/models.py
get_valuemethod toJobmodel for retrieving report values.download_file_from_uri.py
Change stored_base64 to list in download_file_from_uri.api_app/analyzers_manager/observable_analyzers/download_file_from_uri.py
stored_base64to a list for multiple content storage.any_compare.py
Simplify AnyCompare pivot logic with get_value method.api_app/pivots_manager/pivots/any_compare.py
_get_valuemethod withget_valuefrom related reports.should_runmethod logic.abuse_submitter.py
Add exception handling for missing parent job in abuse_submitter.api_app/connectors_manager/connectors/abuse_submitter.py
AnalyzerRunException.0034_changed_resubmitdownloadedfile_playbook_to_execute.py
Change playbook for ResubmitDownloadedFile pivot.api_app/pivots_manager/migrations/0034_changed_resubmitdownloadedfile_playbook_to_execute.py
ResubmitDownloadedFilepivot.load_file.py
Update get_value_to_pivot_to method for list handling.api_app/pivots_manager/pivots/load_file.py
get_value_to_pivot_tomethod to handle lists.job.py
Add is_sample field and fix ZIP MIME type check.api_app/serializers/job.py
is_samplefield toJobSerializer.vt3_get.py
Add update method to VirusTotalv3 analyzer.api_app/analyzers_manager/observable_analyzers/vt/vt3_get.py
updatemethod toVirusTotalv3analyzer.signals.py
Add signal handler for playbooks_choice changes in PivotConfig.api_app/pivots_manager/signals.py
playbooks_choiceofPivotConfig.classes.py
Add logging for ingestor start and finish.api_app/ingestors_manager/classes.py
before_runandafter_runmethods.models.py
Add LNK MIME type to MimeTypes enum.api_app/analyzers_manager/models.py
LNKMIME type toMimeTypesenum._calculate_from_filenamemethod for LNK files.malware_bazaar.py
Simplify logging in get_recent_samples method.api_app/ingestors_manager/ingestors/malware_bazaar.py
get_recent_samples.droidlysis.py
Increase max_tries for DroidLysis analyzer.api_app/analyzers_manager/file_analyzers/droidlysis.py
max_triesfor HTTP request polling.DroidLysisanalyzer.permissions.py
Add PivotActionsPermission for pivot actions.api_app/pivots_manager/permissions.py
PivotActionsPermissionclass for pivot actions.permissions.py
Add isPluginActionsPermission for plugin actions.api_app/permissions.py
isPluginActionsPermissionclass for plugin actions.mwdb_scan.py
Add error handling for MWDB queries in mwdb_scan.api_app/analyzers_manager/file_analyzers/mwdb_scan.py
ObjectNotFoundErrorinrunmethod.secrets.py
Exclude AWS_REGION from secret retrieval in get_secret.threat_matrix/secrets.py
AWS_REGIONfrom secret retrieval.tasks.py
Simplify execute_ingestor function call.threat_matrix/tasks.py
execute_ingestorfunction call.signals.py
Use rest_framework ValidationError in signals.api_app/playbooks_manager/signals.py
ValidationErrorimport to userest_framework.observables.js
Update observableValidators for default value handling.frontend/src/utils/observables.js
observableValidatorsto handle default values.miscConst.js
Add HTTPMethods constant for HTTP method types.frontend/src/constants/miscConst.js
HTTPMethodsconstant for HTTP method types.pluginConst.js
Add AllPluginSupportedTypes constant for plugin types.frontend/src/constants/pluginConst.js
AllPluginSupportedTypesconstant for plugin types.utils.js
Add is_sample property to job node creation.frontend/src/components/investigations/flow/utils.js
is_sampleproperty to job node creation.ScanForm.jsx
Refactor ScanForm component for improved readability.frontend/src/components/scan/ScanForm.jsx
ScanFormcomponent for improved readability.AnalyzerConfigForm.jsx
New Analyzer Configuration Form Componentfrontend/src/components/plugins/forms/AnalyzerConfigForm.jsx
AnalyzerConfigFormfor configuringanalyzers.
PlaybookConfigForm.jsx
New Playbook Configuration Form Componentfrontend/src/components/plugins/forms/PlaybookConfigForm.jsx
PlaybookConfigFormfor configuringplaybooks.
visualizers.
PivotConfigForm.jsx
New Pivot Configuration Form Componentfrontend/src/components/plugins/forms/PivotConfigForm.jsx
PivotConfigFormfor configuring pivots.PluginData.jsx
Support for Ingestors in Plugin Data Componentfrontend/src/components/user/config/PluginData.jsx
PluginDatacomponent to include ingestors in pluginconfigurations.
15 files
0025_ingestor_config_virustotal_example_query.py
Added migration for VirusTotal example query ingestor configurationapi_app/ingestors_manager/migrations/0025_ingestor_config_virustotal_example_query.py
0120_alter_analyzerconfig_not_supported_filetypes_and_more.py
Updated AnalyzerConfig model filetype fields and choicesapi_app/analyzers_manager/migrations/0120_alter_analyzerconfig_not_supported_filetypes_and_more.py
AnalyzerConfigmodel fields for supported and not supportedfiletypes.
0033_pivot_config_extractedonenotefiles.py
Added migration for ExtractedOneNoteFiles pivot configurationapi_app/pivots_manager/migrations/0033_pivot_config_extractedonenotefiles.py
ExtractedOneNoteFilespivot configuration.0130_analyzer_config_nvd_cve.py
Added migration for NVD_CVE analyzer configurationapi_app/analyzers_manager/migrations/0130_analyzer_config_nvd_cve.py
NVD_CVEanalyzer configuration.0052_playbook_config_uris.py
Added migration for Uris playbook configurationapi_app/playbooks_manager/migrations/0052_playbook_config_uris.py
Urisplaybook configuration..prettierignore
Added .prettierignore file for artifact exclusionfrontend/.prettierignore
.prettierignorefile to ignore specific artifacts.__init__.py
Add import for a_secrets module in settings.threat_matrix/settings/init.py
a_secretsmodule.a_secrets.py
Add a_secrets module for AWS region configuration.threat_matrix/settings/a_secrets.py
a_secretsmodule for AWS region configuration.mail.py
Change AWS_REGION import to use a_secrets.threat_matrix/settings/mail.py
AWS_REGIONto usea_secrets.db.py
Change AWS_REGION import to use a_secrets.threat_matrix/settings/db.py
AWS_REGIONto usea_secrets.celery.py
Add task_default_priority to Celery configuration.threat_matrix/celery.py
task_default_priorityto Celery configuration.celery_ingestor.sh
Update Celery worker arguments for AWS SQS support.docker/entrypoints/celery_ingestor.sh
celery_default.sh
Update Celery worker arguments for AWS SQS support.docker/entrypoints/celery_default.sh
celery_long.sh
Update Celery worker arguments for AWS SQS support.docker/entrypoints/celery_long.sh
celery_local.sh
Update Celery worker arguments for AWS SQS support.docker/entrypoints/celery_local.sh
19 files
test_api.py
Added tests for job rescan functionality and permissionstests/api_app/test_api.py
test_mixins.py
Added tests for VirusTotal API mixinstests/api_app/test_mixins.py
VirusTotalv3BaseMixinandVirusTotalv3AnalyzerMixin.test_views.py
Added tests for analyzer configuration CRUD operationstests/api_app/analyzers_manager/test_views.py
configurations.
test_views.py
Added tests for pivot configuration CRUD operationstests/api_app/pivots_manager/test_views.py
test_doc_info.py
Added tests for DocInfo analyzer covering multiple scenariostests/api_app/analyzers_manager/file_analyzers/test_doc_info.py
DocInfoanalyzer focusing on Follina, macros, CVEs,and URLs.
test_classes.py
Update test classes for new file types and configurations.tests/api_app/analyzers_manager/test_classes.py
__init__.py
Add job creation and analysis methods in tests.tests/init.py
test_strings_info.py
Add test case for StringsInfo analyzer.tests/api_app/analyzers_manager/file_analyzers/test_strings_info.py
StringsInfoanalyzer.test_serializers.py
Add tests for PivotConfig creation with plugin config.tests/api_app/pivots_manager/test_serializers.py
PivotConfigwith and without pluginconfiguration.
test_iocextract.py
Add test case for IocExtract analyzer.tests/api_app/analyzers_manager/file_analyzers/test_iocextract.py
IocExtractanalyzer.test_boxjs.py
Add test case for BoxJS analyzer.tests/api_app/analyzers_manager/file_analyzers/test_boxjs.py
BoxJSanalyzer.test_onenote_info.py
Add test case for OneNoteInfo analyzer.tests/api_app/analyzers_manager/file_analyzers/test_onenote_info.py
OneNoteInfoanalyzer.test_pdf_info.py
Add test case for PDFInfo analyzer.tests/api_app/analyzers_manager/file_analyzers/test_pdf_info.py
PDFInfoanalyzer.test_nvd_cve.py
Add test cases for NVDDetails analyzer.tests/api_app/analyzers_manager/observable_analyzers/test_nvd_cve.py
NVDDetailsanalyzer.test_lnk_info.py
Add test case for LnkInfo analyzer.tests/api_app/analyzers_manager/file_analyzers/test_lnk_info.py
LnkInfoanalyzer.observables.test.js
Update observable validators tests with new cases.frontend/tests/utils/observables.test.js
mock.js
Add new mock implementations for auth and organization stores.frontend/tests/mock.js
JobActionBar.test.jsx
Update JobActionsBar tests with new rescan logic.frontend/tests/components/jobs/result/utils/JobActionBar.test.jsx
JobActionsBarwith new rescan logic.PlaybookConfigForm.test.jsx
Tests for Playbook Configuration Form Componentfrontend/tests/components/plugins/types/forms/PlaybookConfigForm.test.jsx
PlaybookConfigFormcomponent.7 files
test_auth.py
Ensure newline consistency in test_auth.py.tests/auth/test_auth.py
environment.js
Reformat THREATMATRIX_DOCS_URL for consistency.frontend/src/constants/environment.js
THREATMATRIX_DOCS_URLfor consistency.verify-email.html
Improve formatting in verify-email.html template.authentication/templates/authentication/emails/verify-email.html
reset-password.html
Improve formatting in reset-password.html template.authentication/templates/authentication/emails/reset-password.html
duplicate-email.html
Improve formatting in duplicate-email.html template.authentication/templates/authentication/emails/duplicate-email.html
base.html
Improve formatting in base email template.authentication/templates/authentication/emails/base.html
UserMenu.jsx
Remove unused import from UserMenu component.frontend/src/layouts/widgets/UserMenu.jsx
UserMenucomponent.1 files
CHANGELOG.md
Changelog formatting and content improvements.github/CHANGELOG.md
71 files
package-lock.json
...frontend/package-lock.json
...
AnalyzerConfigForm.test.jsx
...frontend/tests/components/plugins/types/forms/AnalyzerConfigForm.test.jsx
...
PivotConfigForm.test.jsx
...frontend/tests/components/plugins/types/forms/PivotConfigForm.test.jsx
...
pluginsMultiSelectDropdownInput.jsx
...frontend/src/components/common/form/pluginsMultiSelectDropdownInput.jsx
...
pluginActionsButtons.jsx
...frontend/src/components/plugins/types/pluginActionsButtons.jsx
...
RuntimeConfigurationModal.jsx
...frontend/src/components/scan/utils/RuntimeConfigurationModal.jsx
...
runtimeConfigurationInput.jsx
...frontend/src/components/common/form/runtimeConfigurationInput.jsx
...
AppHeader.test.jsx
...frontend/tests/layouts/AppHeader.test.jsx
...
pull_request_template.md
....github/pull_request_template.md
...
PluginsContainer.jsx
...frontend/src/components/plugins/PluginsContainer.jsx
...
AppHeader.jsx
...frontend/src/layouts/AppHeader.jsx
...
InvestigationFlow.test.jsx
...frontend/tests/components/investigations/flow/InvestigationFlow.test.jsx
...
pluginActionsButtons.test.jsx
...frontend/tests/components/plugins/types/pluginActionsButtons.test.jsx
...
package.json
...frontend/package.json
...
PluginsContainers.test.jsx
...frontend/tests/components/plugins/PluginsContainers.test.jsx
...
traefik_prod.yml
...docker/traefik_prod.yml
...
pluginTableColumns.jsx
...frontend/src/components/plugins/types/pluginTableColumns.jsx
...
TLPSelectInput.jsx
...frontend/src/components/common/form/TLPSelectInput.jsx
...
ScanConfigSelectInput.jsx
...frontend/src/components/common/form/ScanConfigSelectInput.jsx
...
project-requirements.txt
...requirements/project-requirements.txt
...
pluginsApi.jsx
...frontend/src/components/plugins/pluginsApi.jsx
...
Home.jsx
...frontend/src/components/home/Home.jsx
...
JobActionBar.jsx
...frontend/src/components/jobs/result/bar/JobActionBar.jsx
...
CODE_OF_CONDUCT.md
....github/CODE_OF_CONDUCT.md
...
api.jsx
...frontend/src/utils/api.jsx
...
SaveAsPlaybooksForm.jsx
...frontend/src/components/jobs/result/bar/SaveAsPlaybooksForm.jsx
...
ci.override.yml
...docker/ci.override.yml
...
Dockerfile
...integrations/malware_tools_analyzers/Dockerfile
...
OrgConfig.jsx
...frontend/src/components/organization/OrgConfig.jsx
...
jobTableColumns.jsx
...frontend/src/components/jobs/table/jobTableColumns.jsx
...
investigationTableColumns.jsx
...frontend/src/components/investigations/table/investigationTableColumns.jsx
...
jobApi.jsx
...frontend/src/components/jobs/result/jobApi.jsx
...
compose.yml
...integrations/phoneinfoga/compose.yml
...
ScanForm.advanced.test.jsx
...frontend/tests/components/scan/ScanForm/ScanForm.advanced.test.jsx
...
useOrganizationStore.jsx
...frontend/src/stores/useOrganizationStore.jsx
...
MyOrgPage.jsx
...frontend/src/components/organization/MyOrgPage.jsx
...
CustomJobNode.jsx
...frontend/src/components/investigations/flow/CustomJobNode.jsx
...
scanApi.jsx
...frontend/src/components/scan/scanApi.jsx
...
default.yml
...docker/default.yml
...
postgres.override.yml
...docker/postgres.override.yml
...
SECURITY.md
....github/SECURITY.md
...
release_template.md
....github/release_template.md
...
GuideWrapper.jsx
...frontend/src/components/GuideWrapper.jsx
...
traefik_local.yml
...docker/traefik_local.yml
...
PluginWrapper.jsx
...frontend/src/components/plugins/types/PluginWrapper.jsx
...
compose.yml
...integrations/cyberchef/compose.yml
...
pull_request_automation.yml
....github/workflows/pull_request_automation.yml
...
issue_template.md
....github/ISSUE_TEMPLATE/issue_template.md
...
Pivots.jsx
...frontend/src/components/plugins/types/Pivots.jsx
...
new_connector.md
....github/ISSUE_TEMPLATE/new_connector.md
...
new_analyzer.md
....github/ISSUE_TEMPLATE/new_analyzer.md
...
ScanForm.observable.test.jsx
...frontend/tests/components/scan/ScanForm/requests/ScanForm.observable.test.jsx
...
MultipleObservablesModal.jsx
...frontend/src/components/scan/utils/MultipleObservablesModal.jsx
...
new_ingestor.md
....github/ISSUE_TEMPLATE/new_ingestor.md
...
test-requirements.txt
...requirements/test-requirements.txt
...
new_playbook.md
....github/ISSUE_TEMPLATE/new_playbook.md
...
new_visualizer.md
....github/ISSUE_TEMPLATE/new_visualizer.md
...
redis.override.yml
...docker/redis.override.yml
...
Ingestors.test.jsx
...frontend/tests/components/plugins/types/Ingestors.test.jsx
...
TagSelectInput.jsx
...frontend/src/components/common/form/TagSelectInput.jsx
...
test.multi-queue.override.yml
...docker/test.multi-queue.override.yml
...
threat_matrix_bi.json
...configuration/elastic_search_mappings/threat_matrix_bi.json
...
test.flower.override.yml
...docker/test.flower.override.yml
...
compose-tests.yml
...integrations/malware_tools_analyzers/compose-tests.yml
...
compose-tests.yml
...integrations/tor_analyzers/compose-tests.yml
...
compose.yml
...integrations/tor_analyzers/compose.yml
...
FUNDING.yml
....github/FUNDING.yml
...
compose.yml
...integrations/malware_tools_analyzers/compose.yml
...
test.override.yml
...docker/test.override.yml
...
flower.override.yml
...docker/flower.override.yml
...
compose-tests.yml
...integrations/cyberchef/compose-tests.yml
...