Provider intermittently fails to initiate MikroTik API

[peteleblond]

Using provider version 0.6.0 and Terraform 1.0.0 & 1.0.1 it appears that the provider intermittently fails to initiate MikroTik API when running (and re-running) a terraform plan or terraform apply. This appears to happen intermittently even if the code remains totally unchanged. Using tcpdump, I've confirmed that API traffic can be seen when the terraform operation suceeds, but there is no API connection attempt when the this problem is encountered.

Example error message (for terraform plan):

2021-07-06T20:53:55.762Z [ERROR] plugin.(*GRPCProvider).ReadResource: error="rpc error: code = Unavailable desc = transport is closing"
2021-07-06T20:53:57.144Z [ERROR] plugin.(*GRPCProvider).UpgradeResourceState: error="rpc error: code = Canceled desc = context canceled"
│ Error: Plugin did not respond
│ 
│   with mikrotik_script.script_startup_config,
│   on mikrotik-config.tf line 21, in resource "mikrotik_script" "script_startup_config":
│   21: resource "mikrotik_script" "script_startup_config" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The plugin logs may
│ contain more details.
╵

Stack trace from the terraform-provider-mikrotik_v0.6.0 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x666656]

goroutine 37 [running]:
github.com/go-routeros/routeros.(*Client).RunArgs(0x0, 0xc000532020, 0x2, 0x2, 0x1, 0x1, 0xc000532020)
        github.com/go-routeros/routeros@v0.0.0-20190719172022-0819accf8221/run.go:21 +0x26
github.com/ddelnano/terraform-provider-mikrotik/client.Mikrotik.FindScript(0xc0004ce1a0, 0x13, 0xc0003e6118, 0x5, 0xc0004ce1c0, 0x15, 0x0, 0xc0004ce180, 0x1b, 0x1, ...)
        github.com/ddelnano/terraform-provider-mikrotik/client/script.go:106 +0x1e0
github.com/ddelnano/terraform-provider-mikrotik/mikrotik.resourceScriptRead(0xc00024a5b0, 0xf98ea0, 0xc0004c8460, 0xc00024a5b0, 0x0)
        github.com/ddelnano/terraform-provider-mikrotik/mikrotik/resource_script.go:91 +0x117
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc00040d170, 0xc00007d9f0, 0xf98ea0, 0xc0004c8460, 0xc0000aae00, 0x0, 0x0)
        github.com/hashicorp/terraform-plugin-sdk@v1.15.0/helper/schema/resource.go:460 +0x119
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ReadResource(0xc0000aac50, 0x11fd6e0, 0xc000363380, 0xc0003574a0, 0xc0000aac50, 0xc000363380, 0xc000345b78)
        github.com/hashicorp/terraform-plugin-sdk@v1.15.0/internal/helper/plugin/grpc_provider.go:525 +0x3d8
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ReadResource_Handler(0xf71820, 0xc0000aac50, 0x11fd6e0, 0xc000363380, 0xc000357440, 0x0, 0x11fd6e0, 0xc000363380, 0xc0000a6000, 0xf6)
        github.com/hashicorp/terraform-plugin-sdk@v1.15.0/internal/tfplugin5/tfplugin5.pb.go:3269 +0x217
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0000ae900, 0x12075c0, 0xc000001800, 0xc0000ed500, 0xc000186c60, 0x18de570, 0x0, 0x0, 0x0)
        google.golang.org/grpc@v1.27.1/server.go:1024 +0x501
google.golang.org/grpc.(*Server).handleStream(0xc0000ae900, 0x12075c0, 0xc000001800, 0xc0000ed500, 0x0)
        google.golang.org/grpc@v1.27.1/server.go:1313 +0xd3d
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000a4460, 0xc0000ae900, 0x12075c0, 0xc000001800, 0xc0000ed500)
        google.golang.org/grpc@v1.27.1/server.go:722 +0xa1
created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/grpc@v1.27.1/server.go:720 +0xa1

Error: The terraform-provider-mikrotik_v0.6.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.`

Other relevant info:

In my configuration, Terraform is only configured to manage MirktoTik resources mikrotik_script and mikrotik_scheduler.

Provider config:

tls = false
insecure = true

[ddelnano]

Hey @peteleblond, thanks for the report.

I upgraded my terraform version to v1.0.1, but I haven't been able to reproduce it. It appears that this script code is not handling errors before running a command (here and here) :(

Are you able to compile the provider yourself? If so, please apply this diff with git apply and then reproduce the issue so we can get a better idea of what the error is

diff --git a/client/scheduler.go b/client/scheduler.go
index bafc686..11896e2 100644
--- a/client/scheduler.go
+++ b/client/scheduler.go
@@ -16,6 +16,11 @@ type Scheduler struct {

 func (client Mikrotik) FindScheduler(name string) (*Scheduler, error) {
        c, err := client.getMikrotikClient()
+
+       if err != nil {
+               return nil, err
+       }
+
        cmd := []string{"/system/scheduler/print", "?name=" + name}
        log.Printf("[INFO] Running the mikrotik command: `%s`", cmd)
        r, err := c.RunArgs(cmd)
@@ -37,6 +42,10 @@ func (client Mikrotik) FindScheduler(name string) (*Scheduler, error) {
 func (client Mikrotik) DeleteScheduler(name string) error {
        c, err := client.getMikrotikClient()

+       if err != nil {
+               return err
+       }
+
        scheduler, err := client.FindScheduler(name)

        if err != nil {
@@ -53,6 +62,10 @@ func (client Mikrotik) DeleteScheduler(name string) error {
 func (client Mikrotik) CreateScheduler(s *Scheduler) (*Scheduler, error) {
        c, err := client.getMikrotikClient()

+       if err != nil {
+               return nil, err
+       }
+
        cmd := Marshal("/system/scheduler/add", s)

        log.Printf("[INFO] Running the mikrotik command: `%s`", cmd)
diff --git a/client/script.go b/client/script.go
index 9885128..2e8fd59 100644
--- a/client/script.go
+++ b/client/script.go
@@ -21,6 +21,9 @@ func (s *Script) Policy() []string {

 func (client Mikrotik) CreateScript(name, owner, source string, policies []string, dontReqPerms bool) (*Script, error) {
        c, err := client.getMikrotikClient()
+       if err != nil {
+               return nil, err
+       }

        policiesString := strings.Join(policies, ",")
        nameArg := fmt.Sprintf("=name=%s", name)
@@ -85,6 +88,9 @@ func (client Mikrotik) UpdateScript(name, owner, source string, policy []string,

 func (client Mikrotik) DeleteScript(name string) error {
        c, err := client.getMikrotikClient()
+       if err != nil {
+               return err
+       }

        script, err := client.FindScript(name)

@@ -101,6 +107,10 @@ func (client Mikrotik) DeleteScript(name string) error {

 func (client Mikrotik) FindScript(name string) (*Script, error) {
        c, err := client.getMikrotikClient()
+
+       if err != nil {
+               return nil, err
+       }
        cmd := []string{"/system/script/print", "?name=" + name}
        log.Printf("[INFO] Running the mikrotik command: `%s`", cmd)
        r, err := c.RunArgs(cmd)

If that will be difficult then we can get that change merged into master, create a new release and have you try it again.

This code also creates a new connection for each function called, which is inefficient and I believe is leaking sockets. How many resources are in this terraform configuration?

[peteleblond]

Hi @ddelnano

Thanks for checking this out. Great work with this Provider, it's going to be immensely useful!

Currently, there are only 2 resources in my TF config, one of each type mikrotik_script and mikrotik_scheduler.

I'm afraid I'm not sure how to compile this myself, though I'll try and look it up when I get a chance. If you could get the change merged into master I'll test again as soon as I can.

[ddelnano]

I have that change included in v0.6.2. Please install v0.6.2 (it should be pushed to the terraform registry in the next hour or so), reproduce the problem and attach your logs again and hopefully we will have a better clue 🤞

[peteleblond]

Sorry, I might have the wrong end of the stick here, but v0.6.2 doesn't seem to be in the TF registry yet, is that expected? I was having a play with filesystem_mirror by cloning the code tagged v0.6.2, but I'm back to compiling it again, which I haven't yet figured out!

Am I missing something?

[ddelnano]

Sorry, it is now. I forgot to publish the release on github, which is why it was hidden.

[ddelnano]

@peteleblond would appreciate if you get a chance to try out the latest release.