Pimatic gets killed when Shellsensor requests data from not reachable remote partner

---

Hi all,
in the past I thought I catched the problem see here, but there are still problem…
I am sharing my local detected weather data with my neighbor. I the past I handled this by updating my data to https://opensensemap.org. Since I got many error messages during data upload and my neighbor during getting the data I stopped to use the service. I create a user in my system who is able to read variables. On the neighbor system I created some Shellsensors like this:

    {
      "attributeName": "abruf-wetterdaten-aussentemperatur",
      "attributeType": "number",
      "attributeUnit": "°C",
      "command": "timeout 10s sh /home/pi/Homation/abrufWetterdaten.sh temperatur",
      "interval": 60000,
      "echo": {
        "additionalNames": []
      },
      "id": "abruf-wetterdaten-aussentemperatur",
      "name": "Abruf Wetterdaten Außentemperatur",
      "class": "ShellSensor"
    }

The part for the temp looks like this within the script:

curl -s -u $user:$pwd $addr/api/variables/aussenbedingungen.temperatur | jq -r .variable.value

This solution works better now and as long as my system is reachable everything work fine, but if not (e.g. for maintenance reason on my system) the pimatic service of my neighbor still stops to work. I tried to handle it by testing if my system is reachable if curl -Is www.google.de | grep "200 OK" >/dev/null; then before executing the curl to my api, but it ends in the same result -> death of his pimatic service.

I really have no idea what could be the problem. Hope you guys can help.

EDIT: I forgot something. By starting the system with sudo pimatic.js the system stops to start with following entrie:

19:42:18.043 [pimatic-echo] started hue emulator on port 80

I doubt the echo plugin causing the problem, but something after this.
BTW, why the pimatic-daemon.log logs less than the terminal does? The pimatic-daemon.log logs only this during the restart procedure:

19:33:05.087 2019-03-11 Monday
19:33:13.892 [pimatic] info: Starting pimatic version 0.9.46
19:33:13.901 [pimatic] info: Node.js version 4.8.3
19:33:13.903 [pimatic] info: OpenSSL version 1.0.2k
19:33:18.920 [pimatic, ppm] info: 2.15.12
19:33:18.985 [pimatic] info: Loading plugin: "pimatic-echo" (0.5.3)
19:33:20.951 [pimatic] info: Loading plugin: "pimatic-cron" (0.8.8)
19:33:21.532 [pimatic] info: Loading plugin: "pimatic-mobile-frontend" (0.9.13)
19:33:22.419 [pimatic] info: Loading plugin: "pimatic-iframe" (0.3.1)
19:33:22.468 [pimatic] info: Loading plugin: "pimatic-sunrise" (0.9.3)
19:33:22.777 [pimatic] info: Loading plugin: "pimatic-gpio" (0.9.0)

@ortin is there any log data on your neighbors pimatic?

Log are there, but unfortunately nothing is logged in the moment of death.

@ortin are you able to post the full script?
How is the error handling coded?
Is it even coded to handle error events?

Might be that an error crashes the shell sensor.

@gleno0h Have you installed amazing dashbutton plugin? If yes, uninstall it and restart pimatic.
For me it helps. See https://forum.pimatic.org/topic/4569/amazing-dash-button-causes-pimatic-to-hang https://forum.pimatic.org/topic/4547/pimatic-0-9-45/38

Here the script:

user=user
pwd=pwd
addr='addr'
type=$1
prevValue=$2

if curl -Is $addr | grep "200 OK" > /dev/null
	then case $type in
			temperatur)	curl -s -u $user:$pwd $addr/api/variables/aussenbedingungen.temperatur		| jq -r .variable.value		;;
			feuchtigkeit)	curl -s -u $user:$pwd $addr/api/variables/aussenbedingungen.luftfeuchtigkeit	| jq -r .variable.value		;;
			druck)		curl -s -u $user:$pwd $addr/api/variables/aussenbedingungen.luftdruck		| jq -r .variable.value		;;
			lux)		curl -s -u $user:$pwd $addr/api/variables/aussenbedingungen.lux			| jq -r .variable.value		;;
			tempRef)	curl -s -u $user:$pwd $addr/api/variables/sonnensensor.temp-low			| jq -r .variable.value		;;
			tempSonne)	curl -s -u $user:$pwd $addr/api/variables/sonnensensor.temp-high		| jq -r .variable.value		;;
		 esac
else return $prevValue
fi

Version with the try to return the previous value. It could be an idea to patch the variable via the API with the last valid value. I skipped this till now.

Today I went to my neighbor and checked the log directly after a crash. I found 60 of following entries with the different types from the script:

Unhandled rejection Error: Error getting attribute value for Abruf Wetterdaten Außentemperatur: parse error: Invalid numeric literal at line 1, column 7

Could this happen, because if my system is down, then there is nothing to get? The variable gets an unknow (resp. in this case unbekannt).
Wouldn’t it be a good idea that pimatic defaultly returns s.th. numeric instead of a string in case of an error occurs for a number defined ShellSensor. -1 or something like this would be better or keep the last value + an error entry in the log.

@Heizelmann: I temporary deleted the dash-button plugin and all the related stuff. Will see in the next days if this helps. Thx for the hint.

@ortin said in Pimatic gets killed when Shellsensor requests data from not reachable remote partner:

I really have no idea what could be the problem. Hope you guys can help.

Can you please post the type of error messages youi get when your pimatic system is not reachable which then caused the neighbor’s pimatic system to stop working? This would be helpful reproduce the cases in an isolated case. I am in contact with the maintainer of the pcap library who needs a test case to reporduce the issue at his end.

The error messages looks like this (see as well in my last post):

Unhandled rejection Error: Error getting attribute value for Abruf Wetterdaten Außentemperatur: parse error: Invalid numeric literal at line 1, column 7

There is nothing more! After 60 (10 for each of the 6 values) of this messages the system hangs.

This I found today, after I cut my remote access:

Unhandled rejection Error: Expected variable abruf-wetterdaten-temperatur-sonne.abruf-wetterdaten-temperatur-sonne to have a numeric value.
    at /home/pi/pimatic-app/node_modules/pimatic/lib/variables-ast-builder.coffee:111:17
    at tryCatcher (/home/pi/pimatic-app/node_modules/pimatic/node_modules/bluebird/js/release/util.js:16:23)

This is the last entry I can find in the log.

Hey all,
I handled the problem now by changing from Shellsenor devicses to a rule based update of local variables of my neighbors system.
The rule executes a shell script which reads via curl the values from my api and writes it to the local.

.
.
.

if curl -Is $remoteAddr | grep "200 OK" >/dev/null
	then
		temperatur=`curl -s -u $remoteUser:$remotePwd $remoteAddr/api/variables/aussenbedingungen.temperatur | jq -r .variable.value`
			curl --silent -X PATCH --header "Content-Type:application/json" --user "$localUser:$localPwd" --data '{"type": "value", "valueOrExpression": "'"$temperatur"'"}'		http://localhost:1234/api/variables/abruf-wetterdaten-aussentemperatur		>/dev/null

		luftfeuchtigkeit=`curl -s -u $remoteUser:$remotePwd $remoteAddr/api/variables/aussenbedingungen.luftfeuchtigkeit | jq -r .variable.value`
			curl --silent -X PATCH --header "Content-Type:application/json" --user "$localUser:$localPwd" --data '{"type": "value", "valueOrExpression": "'"$luftfeuchtigkeit"'"}'	http://localhost:1234/api/variables/abruf-wetterdaten-luftfeuchtigkeit		>/dev/null

		druck=`curl -s -u $remoteUser:$remotePwd $remoteAddr/api/variables/aussenbedingungen.luftdruck | jq -r .variable.value`
			curl --silent -X PATCH --header "Content-Type:application/json" --user "$localUser:$localPwd" --data '{"type": "value", "valueOrExpression": "'"$druck"'"}'				http://localhost:1234/api/variables/abruf-wetterdaten-luftdruck				>/dev/null

		lux=`curl -s -u $remoteUser:$remotePwd $remoteAddr/api/variables/aussenbedingungen.lux | jq -r .variable.value`
			curl --silent -X PATCH --header "Content-Type:application/json" --user "$localUser:$localPwd" --data '{"type": "value", "valueOrExpression": "'"$lux"'"}'				http://localhost:1234/api/variables/abruf-wetterdaten-helligkeit			>/dev/null

		tempRef=`curl -s -u $remoteUser:$remotePwd $remoteAddr/api/variables/sonnensensor.temp-high | jq -r .variable.value`
			curl --silent -X PATCH --header "Content-Type:application/json" --user "$localUser:$localPwd" --data '{"type": "value", "valueOrExpression": "'"$tempRef"'"}'			http://localhost:1234/api/variables/abruf-wetterdaten-temperatur-referenz	>/dev/null

		tempSonne=`curl -s -u $remoteUser:$remotePwd $remoteAddr/api/variables/sonnensensor.temp-low | jq -r .variable.value`
			curl --silent -X PATCH --header "Content-Type:application/json" --user "$localUser:$localPwd" --data '{"type": "value", "valueOrExpression": "'"$tempSonne"'"}'			http://localhost:1234/api/variables/abruf-wetterdaten-temperatur-sonne		>/dev/null
fi

@mwittig: If you like I can provide the script, create a test user for you on my system and you can access remotely. In this way the conditions are the same and we can play little bit. The system of my neighbor hangs after 5-10s when my system was not available. It was easy to reproduce it. Give me a PM if you like to test.

@ortin said in Pimatic gets killed when Shellsensor requests data from not reachable remote partner:

@mwittig: If you like I can provide the script, create a test user for you on my system and you can access remotely. In this way the conditions are the same and we can play little bit. The system of my neighbor hangs after 5-10s when my system was not available. It was easy to reproduce it. Give me a PM if you like to test.

Thank you very much for your support on this and your offer. I’ll have a look into this soon