Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Look into a dedicated scan executor for notification scans #1055

Open
keith-turner opened this issue Oct 17, 2018 · 1 comment
Open

Look into a dedicated scan executor for notification scans #1055

keith-turner opened this issue Oct 17, 2018 · 1 comment

Comments

@keith-turner
Copy link
Contributor

Scanning for notifications is a continuous activity that Fluo does. It may make sense to do one of the following.

  • Have a dedicated thread pool for notifcation scans with a small number of threads like Math.max(1,0.10 * numCores)
  • Have a prioritizer that gives notification scans a lower priority.

Fluo could offer the prioritizer and/or dispatcher AND utilities or docs to configure them.
@keith-turner
Copy link
Contributor Author

For prioritization, Fluo could use the built in prioritizer with scan execution hints.

keith-turner added a commit to keith-turner/accumulo that referenced this issue Feb 19, 2019
@keith-turner
Support dispatching and prioritizing by user scan types
378b4a6 
While working on apache/fluo#1055 I found it cumbersome to configure
Fluo and Accumulo to dispatch Fluo notification scans to a dedicated
executor.  With these changes its much simpler to do that.  Fluo can
set an execution hint  like `scan_type=fluo-ntfy` and Accumulo can be
configured to execute the scan based on the type without changing Fluo
config or source code.

Before these changes, Fluo would have required either a custom
dispatcher or custom Fluo config for Accumulo executors. With these
changes nothing needs to be done in Fluo.
@keith-turner keith-turner mentioned this issue Feb 19, 2019
Merged
keith-turner added a commit to keith-turner/fluo that referenced this issue Feb 20, 2019
@keith-turner
Identify nofification scan as type in execution hint apache#1055
1f30731
keith-turner added a commit that referenced this issue Feb 21, 2019
@keith-turner
Identify nofification scan as type in execution hint #1055 (#1072)
c128be9
keith-turner added a commit to apache/accumulo that referenced this issue Feb 21, 2019
@keith-turner
Support dispatching and prioritizing by user scan types (#972)
4a8df9a 
While working on apache/fluo#1055 I found it cumbersome to configure
Fluo and Accumulo to dispatch Fluo notification scans to a dedicated
executor.  With these changes its much simpler to do that.  Fluo can
set an execution hint  like `scan_type=fluo-ntfy` and Accumulo can be
configured to execute the scan based on the type without changing Fluo
config or source code.

Before these changes, Fluo would have required either a custom
dispatcher or custom Fluo config for Accumulo executors. With these
changes nothing needs to be done in Fluo.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@keith-turner