read logfile with multiple lines?

---

Is it somehow possible to read a logfile containing several lines into variable?
My logfile looks like this

name: Woonkamer 	temp: 19.31 	setptemp: 20.5 	id: 2084864 	mode: Foll 	until: 00:00:00
name: Keuken 	temp: 20.14 	setptemp: 20.0 	id: 2084865 	mode: Foll 	until: 00:00:00
name: Badkamer 	temp: 18.91 	setptemp: 15.0 	id: 2084866 	mode: Foll 	until: 00:00:00
name: Opkamer 	temp: 19.99 	setptemp: 18.0 	id: 2084867 	mode: Foll 	until: 00:00:00
name: Hal 	temp: 20.4 	setptemp: 20.0 	id: 2084868 	mode: Foll 	until: 00:00:00
name: Werkkamer 	temp: 21.38 	setptemp: 21.0 	id: 2158952 	mode: Temp 	until: 19:00:00
name: Slp. Kamer 	temp: 10.18 	setptemp: 10.0 	id: 2158953 	mode: Foll 	until: 00:00:00

I created a logfile reader device with the following specififcations:

    {
      "file": "/home/pi/evohome-client/temp_per_zone.log",
      "attributes": [
        {
          "name": "name",
          "type": "string"
        },
        {
          "name": "temp",
          "type": "number"
        },
        {
          "name": "setptemp",
          "type": "number"
        },
        {
          "name": "id",
          "type": "number"
        },
        {
          "name": "mode",
          "type": "string"
        },
        {
          "name": "time",
          "type": "string"
        }
      ],
      "lines": [
        {
          "match": "name: (.*)\\btemp",
          "name": "$1"
        },
        {
          "match": "temp: (\\d*.\\d*)",
          "temp": "$1"
        },
        {
          "match": "setptemp: (\\d*.\\d*)",
          "setptemp": "$1"
        },
        {
          "match": "id: (\\d*)",
          "id": "$1"
        },
        {
          "match": "mode: (\\w*)",
          "mode": "$1"
        },
        {
          "match": "until: (\\d*:\\d*:\\d*)",
          "time": "$1"
        }
      ],
      "xAttributeOptions": [],
      "id": "evohome-zones",
      "name": "evohome-zones",
      "class": "LogWatcher"
    }

This device only reads the first line.
How can I read all lines, from this logfile? Is this possible?

@jee67 said in read logfile with multiple lines?:

How can I read all lines, from this logfile? Is this possible?

How do you update update the log file?
Note, other processes writing to the file need to open it in “append” mode!

@mwittig
I think I figured it out. The file is created every 5 minutes as a whole, it’s not that lines are added. Somewhere I read the log reader only reads the last line and that’s exactly what happens. It looks like logreader can’t handle multiple lines. So I probably have to rethink the way I can reach my goal, which is filling up to 40 variable in one go. Maby changing my script in a way that only one line is created in the log file.
I hoped logreader was able to use the regex \n or \W expression.

@jee67 said in read logfile with multiple lines?:

Somewhere I read the log reader only reads the last line and that’s exactly what happens.

It should work if you simply append the sensor data to the log file, e.g. using >> as part of a shell script. The log reader will pick sensor data line by line using the defined match filters and will set attributes accordingly. If you want to keep the log file short, i.e., just one line for each sensor you can use truncate --size 0 to empty the file after each output cycle.

https://linux.die.net/man/1/truncate

@mwittig
I changed the script in order to have all data on one line. But a new problem seems to arise. The match line with the regex expression is so long that Pimatic seems to crash. I could only restart Pimatic after manually deleting the device in the config file. Is there a limitation in the size of the match line logwatcher can process?

@jee67 said in read logfile with multiple lines?:

@mwittig
I changed the script in order to have all data on one line. But a new problem seems to arise. The match line with the regex expression is so long that Pimatic seems to crash. I could only restart Pimatic after manually deleting the device in the config file. Is there a limitation in the size of the match line logwatcher can process?

My matchline looks like this:

name:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \tname:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \tname:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \tname:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \tname:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \tname:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \tname:(.*)\btemp: (\d*.\d*).*\bsetptemp: (\d*.\d*).*\bid: (.*)\bmode: (\w*).*until: (\d*:\d*:\d*) \t

Should logwatcher be able to process this, or is it to complex?

@jee67 I would leave the log output as it was before and create a logwatcher device for each sensor/room. I have taken your sample data and created a logwatcher device for “woonkamer”. It is yet incomplete, but I am sure you can easily setup the remaining bits and pieces. Once, you have everything in place for “woonkamer” you can easily clone the config using copy/paste with configuration editor if you feel comfortable with this. Use the device editor otherwise

    {
      "file": "/home/pi/evohome-client/temp_per_zone.log",
      "attributes": [
        {
          "name": "temp",
          "type": "number",
          "unit": "°C",
          "acronym": "T"
        },
        {
          "name": "setptemp",
          "type": "number",
          "unit": "°C",
          "acronym": "ST"
        }
      ],
      "lines": [
        {
          "match": "name: Woonkamer\\s+temp: (\\d+\\.\\d+)\\s+setptemp: (\\d+\\.\\d+)",
          "temp": "$1",
          "setptemp": "$2"
        }
      ],
      "xAttributeOptions": [],
      "id": "woonkamer-log-reader",
      "name": "Woonkamer",
      "class": "LogWatcher"
    }

@mwittig
Switched back to my original logfile and I created a logwatcher with a matchline for every line in the log file./ It now works as I wanted. so thanks for the hint.

@jee67 said in read logfile with multiple lines?:

I created a logwatcher with a matchline for every line in the log file

Still sounds confusing and contradictory to my advice. My advice is to create a logwatcher device for each sensor/room and define the match expression for each logwatcher device accordingly (see also my previous post)

@mwittig said in read logfile with multiple lines?:

Still sounds confusing and contradictory to my advice. My advice is to create a logwatcher device for each sensor/room and define the match expression for each logwatcher device accordingly (see also my previous post)

That was my configuration, but for that I have to use a rather complex script and multiple logwatchers. My config.json became quite large and complex too. Everything now works exactly as I want.

I am busy integrating Evohome into Pimatic and I want this as easy as possible (to maintain).

Once I’m satisfied, I will post my Evohome integration on the forum. Since my programming skills for coffee are very limited I choose for scripting instead of a module. Maby anyone else can use the scripts and setup to create a module afterwards.

@jee67 Thanks for the clarification! Your work on Evohome integration sound great. I am looking forward to your report on your findings.