Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertions never seem to fail #51

Open
johntdyer opened this issue Mar 23, 2013 · 7 comments
Open

Assertions never seem to fail #51

johntdyer opened this issue Mar 23, 2013 · 7 comments

Comments

@johntdyer
Copy link

Hey Guys,

So I just started playing with minitest, and so far the idea sounds great but I am having a little trouble...

I was able to get the tests to run, as evident in the logs below, however it doesn't seem like any of my assertions ever fail

Here is my default_test.rb

require 'minitest/spec'
require File.expand_path('../support/helpers', __FILE__)

describe_recipe "prism::default" do
    include Helpers::Prism

    describe "test" do
        ## THIS SHOULD FAIL
        it {     file("/etc/god/conf.d/apache2.god").must_exist }
    end

    describe "voxeo-smanager" do

        it "should have voxeo-smanger running" do
            service("voxeo-smanager").must_be_running
        end

        it "should have voxeo-smanger running" do
            service("voxeo-smanager").must_be_enabled
        end

        it "should respond on control port when running" do
            assert shell_out("curl -sI http://localhost:10086").exitstatus == 0
        end

    end

    describe "voxeo-as" do

        it "should respond via http when running" do
            assert shell_out("curl -sI http://localhost:8080").exitstatus == 1
        end

        it "should have voxeo-as running" do
            service("voxeo-as").must_be_enabled
        end

        it "should have voxeo-ms running" do
            service("voxeo-ms").must_be_enabled
        end
    end

    describe "voxeo-ms" do

        it "should have voxeo-as running" do
            service("voxeo-as").must_not_be_running
        end

        it "should have voxeo-ms running" do
            service("voxeo-ms").must_be_running
        end

        it "should respond on control port when running" do
            assert shell_out("curl -sI http://localhost:10099").exitstatus == 0
        end

    end

    describe "service_report" do
        it "should not have any failures" do
            service_report = shell_out "service_report"
            service_report.stdout.must_not_match(/Failures/) #this is a regex match
        end
    end

end

my helpers.rb

module Helpers
    module Prism
        require 'chef/mixin/shell_out'
        include Chef::Mixin::ShellOut
        include MiniTest::Chef::Assertions
        include MiniTest::Chef::Context
        include MiniTest::Chef::Resources

    end
end

You can see that all my tests always pass

[2013-03-23T02:49:45+00:00] DEBUG: service[nrpe] is running
nrpe            0:off   1:off   2:on    3:on    4:on    5:on    6:off
Shutting down nrpe: [  OK  ]
Starting nrpe: [  OK  ]
[2013-03-23T02:49:46+00:00] INFO: service[nrpe] restarted
[2013-03-23T02:49:46+00:00] INFO: Chef Run complete in 29.398107 seconds
[2013-03-23T02:49:46+00:00] DEBUG: Cleaning the checksum cache
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1u4g80w
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1m6uzar
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1b3rpia
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-30h6lw
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1kyuag4
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1sj4cj2
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1f3ljly
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1lm9jo2
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-13zzk83
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-9t9m09
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-5lmzex
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-avbwll
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-jk0gp6
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-7ud2xb
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1c07crs
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1f8nde1
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-r7k0ck
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-h04wcx
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-16fhew1
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-e8odfw
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1syakew
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-p6im31
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-y7wr1n
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-ylf6nj
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-r1f17p
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-1u4caxi
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-7so71u
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-wix8zj
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-m93nxi
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-4ofkgh
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-s4skzm
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-4tppwo
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-tbx863
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-8m621g
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-mayhyw
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-ecm4g4
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-15ejnul
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-s727qn
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-17f7i6d
[2013-03-23T02:49:46+00:00] DEBUG: Removing unused checksum cache file /var/chef/cache/checksums/chef-file--tmp-chef-rendered-template20130323-4041-milp0k
[2013-03-23T02:49:46+00:00] INFO: Running report handlers
Run options: -v --seed 47050

# Running tests:

recipe::prism::default::service_report#test_0001_should_not_have_any_failures = 0.00 s = true
recipe::prism::default::test#test_0001_anonymous = 0.00 s = true
recipe::prism::default::voxeo-as#test_0001_should_respond_via_http_when_running = 0.00 s = true
recipe::prism::default::voxeo-as#test_0002_should_have_voxeo_as_running = 0.00 s = true
recipe::prism::default::voxeo-as#test_0003_should_have_voxeo_ms_running = 0.00 s = true
recipe::prism::default::voxeo-ms#test_0001_should_have_voxeo_as_running = 0.00 s = true
recipe::prism::default::voxeo-ms#test_0003_should_respond_on_control_port_when_running = 0.00 s = true
recipe::prism::default::voxeo-ms#test_0002_should_have_voxeo_ms_running = 0.00 s = true
recipe::prism::default::voxeo-smanager#test_0001_should_have_voxeo_smanger_running = 0.00 s = true
recipe::prism::default::voxeo-smanager#test_0003_should_respond_on_control_port_when_running = 0.00 s = true
recipe::prism::default::voxeo-smanager#test_0002_should_have_voxeo_smanger_running = 0.01 s = true


Finished tests in 0.066084s, 166.4548 tests/s, 0.0000 assertions/s.

11 tests, 0 assertions, 0 failures, 0 errors, 0 skips
[2013-03-23T02:49:47+00:00] INFO: Report handlers complete
[2013-03-23T02:49:47+00:00] DEBUG: Exiting
[root@prism-berkshelf vagrant-chef-1]#

However some of these tests should absolutely be failing.... . Any idea what the heck I am doing wrong?

@calavera
Copy link
Contributor

At first glance everything looks good. My only guess is that the resource helpers are not returning what they should.

Can you tell us which version of Chef and minitest-chef-handler you're using? That would help to narrow the possibilities.

@calavera
Copy link
Contributor

The version of minitest would also help.

@johntdyer
Copy link
Author

Chef 10.16.2
Minitest-hander cookbook 0.1.2

When I checked the system gems on the VM I see the following

[vagrant@prism-berkshelf cookbooks]$ gem list | grep mini
minitest (3.0.1, 1.6.0)
minitest-chef-handler (0.6.8)

I hope this helps!

-John

-John Dyer
m. +1.407.474.c0214

e. johntdyer@gmail.com

Sent from mobile
On Mar 23, 2013, at 5:40 AM, David Calavera notifications@github.com
wrote:

The version of minitest would also help.


Reply to this email directly or view it on GitHub.<
https://github.com/notifications/beacon/cA0wFlwoaNexlU9ZzfIX4WTgn-MiSlCI-J4unnfxelPmrY8Kifnza7CogDDJB89e.gif

@calavera
Copy link
Contributor

thanks @johntdyer.

For three or four versions, the file resources in Chef were completely broken due to their effort to support windows.

I know this is not what you expect but I'd recommend you to upgrade to the latest version in the series 10.18 if you want to use the resource matchers, everything should work fine then.

If you can't/don't want to upgrade your version of Chef, you can go with straight ruby and do something like this:

it { File.exist?(file_path).should be_true }

That should work regardless your version of Chef.

Please, let me know if any this works for you. Thanks again for taking your time reporting this.

@johntdyer
Copy link
Author

Hey,

So I am not sure if this is in fact the issue... So I set up my test so pretty much ALL of them should fail, especially the file one

require 'minitest/spec'
require File.expand_path('../support/helpers', __FILE__)

describe_recipe "prism::default" do
    include Helpers::Prism

    describe "test" do
        ## THIS SHOULD FAIL
        it { File.exist?("/file/that/doesnt/exist").should be_true }

    end

    describe "voxeo-smanager" do

        it "should have voxeo-smanger running" do
            service("voxeo-smanager").must_be_running
        end

        it "should have voxeo-smanger running" do
            service("voxeo-smanager").must_be_enabled
        end

        it "should respond on control port when running" do
         ## THIS SHOULD FAIL
            assert shell_out("curl -sI http://localhost:10086").exitstatus.should eql 1
        end

    end

    describe "voxeo-as" do

        it "should respond via http when running" do
        ## THIS SHOULD FAIL
            assert shell_out("curl -sI http://localhost:8080").exitstatus.should eql 1
        end

        it "should have voxeo-as running" do
            service("voxeo-as").must_be_enabled
        end

        it "should have voxeo-ms running" do
            service("voxeo-ms").must_be_enabled
        end
    end

    describe "voxeo-ms" do

        it "should have voxeo-as running" do
            service("voxeo-as").must_not_be_running
        end

        it "should have voxeo-ms running" do
            service("voxeo-ms").must_be_running
        end

        it "should respond on control port when running" do
        ## THIS SHOULD FAIL
            assert shell_out("curl -sI http://localhost:10099").exitstatus.should eql 1
        end

    end

    describe "service_report" do
        it "should not have any failures" do
            service_report = shell_out "service_report"
            service_report.stdout.must_not_match(/Failures/) #this is a regex match
        end
    end

end

If you notice its saying I am making 0 assertions

2013-03-24T16:00:52+00:00] INFO: Running report handlers
Run options: -v --seed 3199


# Running tests:


recipe::prism::default::service_report#test_0001_should_not_have_any_failures =
0.00 s =
true

recipe::prism::default::test#test_0001_anonymous =
0.00 s =
true

recipe::prism::default::voxeo-as#test_0001_should_respond_via_http_when_running =
0.00 s =
true

recipe::prism::default::voxeo-as#test_0002_should_have_voxeo_as_running =
0.00 s =
true

recipe::prism::default::voxeo-as#test_0003_should_have_voxeo_ms_running =
0.00 s =
true

recipe::prism::default::voxeo-ms#test_0001_should_have_voxeo_as_running =
0.00 s =
true

recipe::prism::default::voxeo-ms#test_0002_should_have_voxeo_ms_running =
0.00 s =
true

recipe::prism::default::voxeo-ms#test_0003_should_respond_on_control_port_when_running =
0.00 s =
true

recipe::prism::default::voxeo-smanager#test_0003_should_respond_on_control_port_when_running =
0.00 s =
true

recipe::prism::default::voxeo-smanager#test_0002_should_have_voxeo_smanger_running =
0.00 s =
true

recipe::prism::default::voxeo-smanager#test_0001_should_have_voxeo_smanger_running =
0.00 s =
true



Finished tests in 0.051378s, 214.0994 tests/s, 0.0000 assertions/s.


11 tests, 0 assertions, 0 failures, 0 errors, 0 skips

[2013-03-24T16:00:52+00:00] INFO: Report handlers complete
[2013-03-24T16:00:52+00:00] DEBUG: Exiting

Any ideas?

@johntdyer
Copy link
Author

@calavera Any ideas?

@johntdyer
Copy link
Author

Anyone else have any ideas here ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants