Provision Device with custom ThingStream device profile ID gives "JSON error"

[edgecase14]

Subject of the issue

A previous attempt with another device failed because I set up the ThingStream Device profile with Auto Activate disabled. I enabled it, and tried on another ESP32 Thing C Plus (different mac address), and then ran into this error.

Provisioning fails with "JSON error"

A second attempt gives:
HTTP response error 403: Device already registered
And I can see in ThingStream a new Location Thing is created and activated. Plan is L-band + IP, I have filtered only US topics.

Your workbench

• What version of RTK firmware are you running? v3.2 binary from github
• This is a custom L-band setup using ESP32 Thing Plus C + F9P-D9S combo board
• What radios are you using: Bluetooth, WiFi,
• What app are you using to connect over Bluetooth? None - using ESP32 USB serial console
• Are you transmitting NTRIP back to the device? No
• Are there any additional details that may help us help you? Manual key entry works, PointPerfect gets 1-2cm accuracy.

Steps to reproduce

console output:

Menu: PointPerfect Corrections
Device ID: E05A1B595312
Days until keys expire: No keys

1. Use PointPerfect Corrections: Enabled
2. Toggle Auto Key Renewal: Enabled
3. Provision Device
   k) Manual Key Entry
   x) Exit
   3
   Connecting WiFi...
   Using custom token:
   Connecting to: https://api.thingstream.io/ztp/pointperfect/credentials
   JSON error
   Bluetooth broadcasting as: Surveyor Rover-XXXX

Expected behavior

I assume it would download device credentials, and then L-band keys via MQTT.

Actual behavior

"JSON error" on console

[nseidle]

I think this may be related to this bug fix. We were passing the size of a pointer instead of the size of the array (bad programmer!).

However, I'm not entirely sure. Please pull the latest RC code and give it a try. Your provisioning process is different than ours so I'm not sure if its our code or your setup on ThingStream.

[nseidle]

I just re-tested with new custom token and all is well:

[nseidle]

This has been implemented in firmware release v3.4.

[edgecase14]

Still same issue in v3.4

[edgecase14]

Would it be possible to share the ThingStream device profile settings that were tested? I could then try the same settings and see if that gives a different result.

[nseidle]

[nseidle]

I know you're on customer hardware, but would you please do a factory reset? There's something odd. Steps to reproduce (for my own notes):

• Install v3.2 firmware
• Factory reset device
• Obtain keys
• Attempt to update keys

At this point, updating keys will fail due to known bug.

• Install v3.4 firmware (contains bug fix for JSON)
• Attempt to update keys

This fails for reasons unknown.

• Factory reset device
• Obtain keys
• Attempt to update keys

This will work. Presumably because the JSON bug fix is in place but I can't explain why the previous settings from v3.2 prevent a v3.4 after upgrade to work. I suspect because size of the settings file in SPIFFS has gotten a lot smaller (in 3.2, settings files were ~4.5kb, in v3.4 settings file is ~1.8kb).

[nseidle]

Ah, there it is:

When we obtained keys in the previous versions (v3.2 for example) the keys were stored corrupted. So even in the new fixed versions, we have to reset the device to erase the certificate and private keys. Hmmm. Perhaps we can do better.

[edgecase14]

Just tried on 3 devices, v3.4, using USB serial UI:

• factory reset
• configure wifi
• enter custom Device Profile ID ( profile settings match Sparkfun example above except Plan is L-band + IP)
• provision device - new Location Thing is created but get JSON Error
• retry provision - Event log shows "the hardware code ... has already been registered, ... allowed to refetch its credentials"

This did not work when Plan was set to PointPerfect L-band and IP. It works if L-band only, so, this is a big improvement (I can now deploy field trial), but IP is desired since some sites (or areas of a site) don't have L-band reception. There is an Android MQTT client, and also future Sparkfun firmware feature request which both would need an L-band + IP Plan to work.

Also, an existing Location Thing, where the firmware was factory reset, was unable to (re)get it's credentials when Plan is L-band + IP. It works ok when Plan is L-band only.

[nseidle]

Issues:

• JSON failing to connect during key update: Fixed here, the certs were getting corrupt when they were created.
• Units with v3.4 firmware, but bad certs, where still experiencing problems. This was fixed here. So versions v3.4 may need to do a factory reset to clear out the corrupt certs, but v3.5 and later they will be automatically detected bad, removed, and re-generated.

The issue of provisioning failing because the system does not work with PointPerfect IP+L-Band service is a bit different. I suspect the MQTT channels are different enough to cause problems. We'll get that sorted once we add IP based PointPerfect corrections. Please see issue #497. I'm going to close this issue because I think the bulk of the problem has been cleared. Please open additional issues as needed.