Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Make agent default compatible with Bluesky Adaptive #86

Merged
merged 19 commits into from
Feb 6, 2025

Conversation

maffettone
Copy link
Contributor

This does a lot of re-factoring, all with the goal of not augmenting existing functionality at all.

The Agent was split into a BaseAgent that it inherits from and Agent. Then a BlueskyAdaptiveAgent was built that inherits from both this BaseAgent and BlueskyAdaptiveBaseAgent, making as much use of the initial functionality as possible.

In a nutshell, we needed to remove extra args and kwargs from the signature of data movement functions (ask/tell). The objects there, should be attributes of the Agent that can be adjusted via the REST API.

Other components of refactoring:

  • Fix ask in bluesky
  • move original tell to base class, and add bluesky tell
  • refactor out update_models from base tell
  • refactor private training methods into base class, and their deps (essentially checking anything self. in these functions that wasn't declared)
  • Add an unpack_run that attempts learn's approach to pulling data, but will be tiled/dbv2 compat. Digestion is risky, but I tried my best.
  • Add measurement plan that is QServer compat (Json serializable name, args, kwargs)
  • Make bluesky relevat attrs into registered properties, and add server registrations.

- Fix ask in bluesky
- move original tell to base class, and add bluesky tell
- refactor out update_models from base tell
-  refactor private training methods into base class, and their deps (essentially anything self. that wasn't decalred)
- Add an unpack run that attempts learn's approach to pulling data, but will be tiled/dbv2 compat. Digestion is risky.
- Add measurement plan that is QServer compat
- Make bluesky relevat attrs into registered properties, and add server registrations
@maffettone
Copy link
Contributor Author

Should any of this CI be passing? I couldn't get any tests to pass on my main branch locally. Just want to sanity check.

@thomashopkins32
Copy link
Contributor

Should any of this CI be passing? I couldn't get any tests to pass on my main branch locally. Just want to sanity check.

Based on this commit (49752b5) I believe it should be.

Copy link
Contributor

@thomashopkins32 thomashopkins32 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of it looks fine since its basically copy-pasted from the existing agent. I have some suggestions but nothing blocking.

I can approve once its been tested (and all other tests pass).

This package does not have great test coverage yet so I am a bit concerned about breaking changes.

pyproject.toml Outdated Show resolved Hide resolved
src/blop/agent.py Outdated Show resolved Hide resolved
src/blop/agent.py Outdated Show resolved Hide resolved
src/blop/agent.py Outdated Show resolved Hide resolved
src/blop/agent.py Outdated Show resolved Hide resolved
src/blop/agent.py Outdated Show resolved Hide resolved
Comment on lines +611 to +613
# Hope and pray that the digestion function designed for DataFrame can handle the XArray
data: pd.DataFrame = self.digestion(run.primary.data.read(), **self.digestion_kwargs)
return [data.loc[:, key] for key in self.dofs.names], [data.loc[:, key] for key in self.objectives.names]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming this prayer can be answered by unit tests? Even with some mock data, if its simpler that way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on stepping back from the tiled/dbv2 requirements for the CI here, I'm going to hold on producing a test for this. Really trying to show a skeleton for how it can be done without breaking existing workflows.

We can definitely build up a smoke test, but I don't quite understand how data will be captured with the default plans using the queue server and tiled.

@maffettone
Copy link
Contributor Author

What I have learned in this process is:

  1. Bluesky-adaptive desperately needs review on test: reduce pod and communication dependencies bluesky/bluesky-adaptive#43 and movement on dependency bloat.
  2. The CI here will be incompatible with data broker v2.

maffettone and others added 2 commits February 6, 2025 13:08
Co-authored-by: Thomas Hopkins <thomashopkins000@gmail.com>
Copy link
Contributor

@thomashopkins32 thomashopkins32 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thomashopkins32
Copy link
Contributor

I realize I am not a contributor yet so I can't merge

@maffettone maffettone merged commit c0eb914 into NSLS-II:main Feb 6, 2025
6 checks passed
@maffettone
Copy link
Contributor Author

Hopefully this merge makes you a contributor.

@maffettone maffettone deleted the enh-bs-adaptive branch February 6, 2025 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants