-
Notifications
You must be signed in to change notification settings - Fork 44
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Additional tests to prep for first release.
- Loading branch information
Showing
9 changed files
with
204 additions
and
132 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,9 @@ | ||
# Changelog | ||
|
||
- Will be maintained after alpha releases are out of the way. :) | ||
## Next version [???] | ||
|
||
## v1.0.0 [06-02-2016] | ||
|
||
- First version released. | ||
- 100% test coverage. | ||
- Full documentation. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,66 +1,37 @@ | ||
# Contribution Guide | ||
|
||
This guide will be updated with more thorough content, but for now, the best thing you could do to help the project is to create `Journal` packages. | ||
|
||
Journals are used to export the information from experiments to data stores for later inspection. They implement the `Scientist\Journals\Journal` interface: | ||
|
||
```php | ||
<?php | ||
|
||
namespace Scientist\Journals; | ||
|
||
use Scientist\Report; | ||
use Scientist\Experiment; | ||
|
||
/** | ||
* Class Journal | ||
* | ||
* Journals allow the scientist to record experiment results in a | ||
* variety of different ways. | ||
* | ||
* @package \Scientist | ||
*/ | ||
interface Journal | ||
{ | ||
/** | ||
* Dispatch a report to storage. | ||
* | ||
* @param \Scientist\Experiment $experiment | ||
* @param \Scientist\Report $report | ||
* | ||
* @return mixed | ||
*/ | ||
public function report(Experiment $experiment, Report $report); | ||
} | ||
|
||
``` | ||
|
||
When you implement the `report()` method, the registered Journal will receive an instance of the experiment, and the report from the executed experiment. You can interrogate these instances for all kinds of useful information, and ship that information to a data storage platform. | ||
|
||
Here's some useful methods: | ||
|
||
```php | ||
$report->getName(); // Get the experiment name. (string) | ||
$report->getControl(); // Get the Scientist\Result instance for the control. (Scientist\Result) | ||
$report->getTrial('name'); // Get the result instance for a trial by name. (Scientist\Result) | ||
$report->getTrials(); // Get an assoc array of trial result instances (array) | ||
``` | ||
|
||
And on the result instance: | ||
|
||
```php | ||
$result->getValue(); // Get the resulting value of the trial/control callback. (mixed) | ||
$result->isMatch(); // Did the result match the control? Use on trial results. (boolean) | ||
$result->getStartTime(); // Float callable start microtime. (float) | ||
$result->getEndTime(); // Float callable end microtime. (float) | ||
$result->getTime(); // Get the execution microtime. (float) | ||
$result->getStartMemory(); // Get memory usage before calling. (integer) | ||
$result->getEndMemory(); // Get memory usage after calling. (integer) | ||
$result->getMemory(); // Get different in memory usage. (integer) | ||
``` | ||
|
||
If you create a new journal (and a composer package, format scientist-xxx-journal) then please be sure to raise an issue and let me know! I'll update the docs to mentions it. | ||
|
||
Thanks for all your support! | ||
|
||
- Dayle. | ||
The best thing a contributor can do for the project right now, is to submit supporting resources. Here's some great ideas for you. | ||
|
||
## Create Bridge Packages | ||
|
||
Bridges allow Scientist to be used seamlessly with other frameworks and libraries. Learn more about Bridges [in the documentation](https://scientist.readme.io/docs/bridges). | ||
|
||
Please name your package `vendorname/scientist-<platform>-bridge`. | ||
|
||
## Create Journals | ||
|
||
Journals allow the Scientist to export his experiment data to data storage platforms for latest inspection. | ||
|
||
Please name your package `vendorname/scientist-<platform>-journal`. | ||
|
||
See more information [in the documentation](https://scientist.readme.io/docs/journals-1). | ||
|
||
## Create Matchers | ||
|
||
Matchers allow for complex data types to be matched using Scientist. | ||
|
||
Please name your package `vendorname/scientist-<platform>-matcher`. | ||
|
||
See more information [in the documentation](https://scientist.readme.io/docs/matchers-1). | ||
|
||
## Create Interfaces | ||
|
||
Interfaces allow for unique ways to view your experiment data. | ||
|
||
Please name your package `vendorname/scientist-<platform>-interface`. | ||
|
||
See more information [in the documentation](https://scientist.readme.io/docs/interfaces). | ||
|
||
--- | ||
|
||
Thanks for supporting the project! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
<?php | ||
|
||
use Scientist\Intern; | ||
use Scientist\Report; | ||
use Scientist\Experiment; | ||
|
||
class InternTest extends PHPUnit_Framework_TestCase | ||
{ | ||
public function test_that_intern_can_be_created() | ||
{ | ||
$i = new Intern; | ||
$this->assertInstanceOf(Intern::class, $i); | ||
} | ||
|
||
public function test_that_intern_can_run_an_experiment() | ||
{ | ||
$i = new Intern; | ||
$e = new Experiment('test experiment'); | ||
$e->control(function () { return 'foo'; }); | ||
$v = $i->run($e); | ||
$this->assertInstanceOf(Report::class, $v); | ||
$this->assertEquals('foo', $v->getControl()->getValue()); | ||
} | ||
|
||
public function test_that_intern_can_match_control_and_trial() | ||
{ | ||
$i = new Intern; | ||
$e = new Experiment('test experiment'); | ||
$e->control(function () { return 'foo'; }); | ||
$e->trial('bar', function () { return 'foo'; }); | ||
$v = $i->run($e); | ||
$this->assertInstanceOf(Report::class, $v); | ||
$this->assertTrue($v->getTrial('bar')->isMatch()); | ||
} | ||
|
||
public function test_that_intern_can_mismatch_control_and_trial() | ||
{ | ||
$i = new Intern; | ||
$e = new Experiment('test experiment'); | ||
$e->control(function () { return 'foo'; }); | ||
$e->trial('bar', function () { return 'bar'; }); | ||
$v = $i->run($e); | ||
$this->assertInstanceOf(Report::class, $v); | ||
$this->assertFalse($v->getTrial('bar')->isMatch()); | ||
} | ||
|
||
public function test_that_intern_can_match_and_mismatch_control_and_trial() | ||
{ | ||
$i = new Intern; | ||
$e = new Experiment('test experiment'); | ||
$e->control(function () { return 'foo'; }); | ||
$e->trial('bar', function () { return 'foo'; }); | ||
$e->trial('baz', function () { return 'baz'; }); | ||
$v = $i->run($e); | ||
$this->assertInstanceOf(Report::class, $v); | ||
$this->assertTrue($v->getTrial('bar')->isMatch()); | ||
$this->assertFalse($v->getTrial('baz')->isMatch()); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
<?php | ||
|
||
use Scientist\Report; | ||
use Scientist\Laboratory; | ||
|
||
class LaboratoryTest extends PHPUnit_Framework_TestCase | ||
{ | ||
public function test_laboratory_can_be_created() | ||
{ | ||
$l = new Laboratory; | ||
$this->assertInstanceOf(Laboratory::class, $l); | ||
} | ||
|
||
public function test_laboratory_can_run_experiment_with_no_journals() | ||
{ | ||
$v = (new Laboratory) | ||
->experiment('test experiment') | ||
->control(function () { return 'foo'; }) | ||
->trial('trial name', function () { return 'foo'; }) | ||
->run(); | ||
|
||
$this->assertEquals('foo', $v); | ||
} | ||
|
||
public function test_laboratory_can_fetch_report_for_experiment_with_no_journals() | ||
{ | ||
$r = (new Laboratory) | ||
->experiment('test experiment') | ||
->control(function () { return 'foo'; }) | ||
->trial('trial', function () { return 'bar'; }) | ||
->report(); | ||
|
||
$this->assertInstanceOf(Report::class, $r); | ||
$this->assertEquals('foo', $r->getControl()->getValue()); | ||
$this->assertEquals('bar', $r->getTrial('trial')->getValue()); | ||
$this->assertInternalType('float', $r->getControl()->getStartTime()); | ||
$this->assertInternalType('float', $r->getControl()->getEndTime()); | ||
$this->assertInternalType('float', $r->getControl()->getTime()); | ||
$this->assertInternalType('float', $r->getTrial('trial')->getStartTime()); | ||
$this->assertInternalType('float', $r->getTrial('trial')->getEndTime()); | ||
$this->assertInternalType('float', $r->getTrial('trial')->getTime()); | ||
$this->assertInternalType('integer', $r->getControl()->getStartMemory()); | ||
$this->assertInternalType('integer', $r->getControl()->getEndMemory()); | ||
$this->assertInternalType('integer', $r->getControl()->getMemory()); | ||
$this->assertInternalType('integer', $r->getTrial('trial')->getStartMemory()); | ||
$this->assertInternalType('integer', $r->getTrial('trial')->getEndMemory()); | ||
$this->assertInternalType('integer', $r->getTrial('trial')->getMemory()); | ||
$this->assertInternalType('null', $r->getControl()->getException()); | ||
$this->assertInternalType('null', $r->getTrial('trial')->getException()); | ||
$this->assertFalse($r->getTrial('trial')->isMatch()); | ||
} | ||
|
||
public function test_that_exceptions_are_thrown_within_control() | ||
{ | ||
$this->setExpectedException(Exception::class); | ||
|
||
$v = (new Laboratory) | ||
->experiment('test experiment') | ||
->control(function () { throw new Exception; }) | ||
->trial('trial', function () { return 'foo'; }) | ||
->run(); | ||
} | ||
|
||
public function test_that_exceptions_are_swallowed_within_the_trial() | ||
{ | ||
$r = (new Laboratory) | ||
->experiment('test experiment') | ||
->control(function () { return 'foo'; }) | ||
->trial('trial', function () { throw new Exception; }) | ||
->report(); | ||
|
||
$this->assertInstanceOf(Report::class, $r); | ||
$this->assertInternalType('null', $r->getControl()->getException()); | ||
$this->assertInstanceOf(Exception::class, $r->getTrial('trial')->getException()); | ||
} | ||
|
||
public function test_that_control_and_trials_receive_parameters() | ||
{ | ||
$r = (new Laboratory) | ||
->experiment('test experiment') | ||
->control(function ($one, $two) { return $one; }) | ||
->trial('trial', function ($one, $two) { return $two; }) | ||
->report('Panda', 'Giraffe'); | ||
|
||
$this->assertInstanceOf(Report::class, $r); | ||
$this->assertEquals('Panda', $r->getControl()->getValue()); | ||
$this->assertEquals('Giraffe', $r->getTrial('trial')->getValue()); | ||
} | ||
} |