Print panic output on Error log level

[hanzei]

First of all, thanks for creating this awesome library! We at Mattermost use it a lot.

If a plugin panics, the panic message goes to stderr of the plugin. Because the log lines don't have any levelled prefix like [ERROR], the panic message would end up in the Debug level. That makes it hard to spot, why a plugin panicked.

The (hacky) solution is to check if any unparsed lines start with panic:. If that is the case, this line and all unparsed following lines go to the Error level. Most likely, these will be the last log lines anyway before the process dies.

I tried adding a test to cover, but that would require spinning up another process (that then panics) to cover this PR.

[hashicorp-cla]

All committers have signed the CLA.

[hanzei]

@tomhjp Would you mind giving this PR a review?

[hanzei]

Anything I can help with to get a review going?

[streamer45]

@VioletHynes Would your team have some bandwidth to review this one?

[VioletHynes]

Hi there -- this isn't my team's area but I'll try and forward this to the team that owns this. Thanks :)

[streamer45]

Thanks, appreciate it!

[hanzei]

@robmonte Gentle reminder to review the PR 😄

[hanzei]

@robmonte Just wanted to check the process on the PR. Is there anything I can do to speed up the process?

[robmonte]

@hanzei Sorry for the delay since my last comment. At first a concern with this change was that, as you said, it's a little hacky to be based on string prefixes. An alternate solution pull request was opened that you can see here: #319

Providing a PanicHandler might be a "more correct" solution long term and so we were looking into that a bit, however given the widespread use of go-plugin it definitely makes us more cautious with changes. I'm not familiar enough with gRPC as a whole to give that PR a proper review despite it seeming straightforward.

However since the entire log level checker is already based on that same string logic, we think it makes sense to go ahead with this smaller, simpler change for now.

[ghost]

Approving the PR after reviewing the changes.

[hanzei]

Awesome, thanks for merging the PR! Looking forward to the next release 🚀