Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add audit trial length optimizely experiment #82

Merged
merged 3 commits into from
Feb 4, 2025
Merged

feat: Add audit trial length optimizely experiment #82

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b8281e1
feat: Add audit trial length optimizely experiment
ilee2u Jan 31, 2025
7b502d2
fix: corrected logic for gating trial w/ var keys
ilee2u Feb 3, 2025
b195dfb
chore: nit
ilee2u Feb 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions src/data/optimizely.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,22 @@ const getOptimizely = () => {
return instance;
};

const OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_KEY = 'xpert_audit_trial_experiment';
const OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS = {
CONTROL: 'control',
XPERT_AUDIT_14_DAY_TRIAL: 'xpert_audit_14_day_trial',
XPERT_AUDIT_28_DAY_TRIAL: 'xpert_audit_28_day_trial',
MichaelRoytman marked this conversation as resolved.
Show resolved Hide resolved
};

const OPTIMIZELY_PROMPT_EXPERIMENT_KEY = '_cosmo__xpert_gpt_4_0_prompt';
const OPTIMIZELY_PROMPT_EXPERIMENT_VARIATION_KEYS = {
UPDATED_PROMPT: 'updated_prompt',
};

export {
getOptimizely,
OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_KEY,
OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS,
OPTIMIZELY_PROMPT_EXPERIMENT_KEY,
OPTIMIZELY_PROMPT_EXPERIMENT_VARIATION_KEYS,
};
14 changes: 13 additions & 1 deletion src/experiments/experimentHooks.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { getAuthenticatedUser } from '@edx/frontend-platform/auth';
import { useDecision } from '@optimizely/react-sdk';

import { OPTIMIZELY_PROMPT_EXPERIMENT_KEY } from '../data/optimizely';
import { OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_KEY, OPTIMIZELY_PROMPT_EXPERIMENT_KEY } from '../data/optimizely';

// eslint-disable-next-line import/prefer-default-export
export function usePromptExperimentDecision() {
Expand All @@ -14,3 +14,15 @@ export function usePromptExperimentDecision() {

return [decision];
}

// eslint-disable-next-line import/prefer-default-export
export function useAuditTrialExperimentDecision() {
const { userId } = getAuthenticatedUser();

const [decision] = useDecision(
OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_KEY,
{ overrideUserId: userId.toString() }, // This override is just to make sure the userId is up to date.
);

return [decision];
}
14 changes: 13 additions & 1 deletion src/hooks/use-course-upgrade.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ import { useModel } from '@src/generic/model-store'; // eslint-disable-line impo
import { useSelector } from 'react-redux';
import { CourseInfoContext } from '../context';

import { OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS } from '../data/optimizely';
import { useAuditTrialExperimentDecision } from '../experiments';

const millisecondsInOneDay = 24 * 60 * 60 * 1000; // hours*minutes*seconds*milliseconds

/**
Expand Down Expand Up @@ -41,9 +44,18 @@ export default function useCourseUpgrade() {
auditTrial,
} = useSelector(state => state.learningAssistant);

const [decision] = useAuditTrialExperimentDecision();
const { enabled, variationKey } = decision || {};

const upgradeUrl = offer?.upgradeUrl || verifiedMode?.upgradeUrl;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a little unclear on what the instruction to "Update logic here to make upgrade eligible if in experiment variation" meant, since the isUpgradeEligible boolean is already passed into this function. Let me know if I had the right idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theoretically, what you'd want to change is what is returned from this function. I agree that modifying isUpgradeEligible wouldn't make sense.isUpgradeEligible represents whether the learner is in a course mode that makes them eligible to upgrade to a paid seat.

Having said that, for the two variation keys, I don't think you'd want to change anything here, right? In both variations, upgradeable should be true. And backend will handle setting the correct length for the trial based on the variation keys using the Python SDK (i.e. auditTrialLengthDays.

I think what we want to do is NOT render the audit trial experience when the learner is in the off or control variations. What do you think?


if (!isUpgradeEligible || !upgradeUrl) { return { upgradeable: false }; }
if (
!isUpgradeEligible
|| !upgradeUrl
|| !enabled
|| (variationKey !== OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS.XPERT_AUDIT_14_DAY_TRIAL
&& variationKey !== OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS.XPERT_AUDIT_28_DAY_TRIAL)
) { return { upgradeable: false }; }

let auditTrialExpired = false;
let auditTrialDaysRemaining;
Expand Down
28 changes: 28 additions & 0 deletions src/hooks/use-course-upgrade.test.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,12 @@ import { useSelector } from 'react-redux';
import { useModel } from '@src/generic/model-store'; // eslint-disable-line import/no-unresolved
import { CourseInfoProvider } from '../context';
import useCourseUpgrade from './use-course-upgrade';
import { OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS } from '../data/optimizely';

import { useAuditTrialExperimentDecision } from '../experiments';

jest.mock('react-redux', () => ({ useSelector: jest.fn() }));
jest.mock('../experiments', () => ({ useAuditTrialExperimentDecision: jest.fn() }));

const mockedUpgradeUrl = 'https://upgrade.edx/course/test';
const mockedAuditTrialLengthDays = 7;
Expand All @@ -31,6 +35,10 @@ const renderHook = ({
});

useSelector.mockReturnValue(state.learningAssistant);
useAuditTrialExperimentDecision.mockReturnValue([{
enabled: true,
variationKey: OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS.XPERT_AUDIT_14_DAY_TRIAL,
}]);

return rtlRenderHook(
() => useCourseUpgrade(),
Expand All @@ -55,6 +63,26 @@ describe('useCourseUpgrade()', () => {
expect(result.current).toEqual({ upgradeable: false });
});

it('should return { upgradeable: false } if audit trial experiment is not enabled', () => {
const { result } = renderHook({ courseInfo: { isUpgradeEligible: true } });
useAuditTrialExperimentDecision.mockReturnValue([{
enabled: false,
variationKey: OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS.XPERT_AUDIT_14_DAY_TRIAL,
}]);

expect(result.current).toEqual({ upgradeable: false });
});

it('should return { upgradeable: false } is not one of the audit trial experiment treatment keys', () => {
useAuditTrialExperimentDecision.mockReturnValue([{
enabled: false,
variationKey: OPTIMIZELY_AUDIT_TRIAL_LENGTH_EXPERIMENT_VARIATION_KEYS.CONTROL,
}]);
const { result } = renderHook({ courseInfo: { isUpgradeEligible: true } });

expect(result.current).toEqual({ upgradeable: false });
});

it('should return { upgradeable: true } if eligible and upgradeable and no trial info for both offer and verifiedMode urls', () => {
const expected = {
upgradeable: true,
Expand Down