Pimatic-openweather OpenWeatherForecastDevice receives wrong data

---

I’m experiencing some problems with the Pimatic-openweather OpenWeatherForecastDevice. A normal OpenWeatherDevice device works fine, but with the OpenWeatherForecastDevice, I often receive invalid data. Because of the wrong data, “unknown” is often being displayed in the GUI.

I tried different settings, but that didn’t help. I’m also having this problem if I use the settings from the documentation (Forecast 1 day for Geneva). I also deleted the pimatic-openweather module directory and deleted the database, but that didn’t help either.

Here you can see the values being received:

I was wondering if I’m the only one with this problem.

@Pazzie Same problem here. Currently, I don’t get valid forecast. As openweather only advertises an availability of 95% for the free API usage this may happen from time to time, but in my opinion the plugin should ignore bogus data. Unfortunately, the debug log capabilities of pimatic-openweather are somewhat limited. I’ll add some debug messaging to better understand the problem.

EDIT: The problem for me is (most likely the same for you) is that the forecast data record only contains the data set of today, but the plugin only takes data for the forthcoming days (1-3 days depending on the configuration). As part of the initialization of the update cycle some variable are inbitialzed with null or Infinity (min and max) and these values get propagated to the DB. I consider this as a bug - I’ll prepare a fix and discuss the matter @sweetpi

@mwittig Thank you very much for confirming this issue. I agree that it would be better to ignore the invalid data.

There are also packages that have the option to provide an API key. openweather-apis (https://www.npmjs.com/package/openweather-apis), for example, has the “weather.setAPIID” param.The openweathermap package unforunately doesn’t have this option. I edited “/home/pi/pimatic-app/node_modules/pimatic-openweather/node_modules/openweathermap/openweather.js” myself, to add my API key. Was that the right place to do it? It didn’t solve the problem with the invalid data.

This is what Openweathermap says about the API key: "To get access to weather API you need an API key whatever account you chose from Free to Enterprise.

We keep right to not to process API requests without API key."

Sorry for the interruption…is mysql database now working?

@temp Please don’t hijack this thread. You can use the chat tool or create a separate post for the matter. Thanks

@Pazzie said:

I edited “/home/pi/pimatic-app/node_modules/pimatic-openweather/node_modules/openweathermap/openweather.js” myself, to add my API key. Was that the right place to do it? It didn’t solve the problem with the invalid data.

Yes, that the right place, however, I think the problem right now is, that no forecast data is available for some reasons. Instead of returnng an error the weather service returns a record with the weather data of today. See EDIT of my earlier post above.

Filed issue report - https://github.com/josecastroleon/pimatic-openweather/issues/16

@mwittig Thank you very much for investigating this and filing the issue report. If you’ve got questions for me, that could help to solve the problem, feel free to ask.

@Pazzie I have filed a PR which partially resolves the issue. If you are familiar with installing packages from git you can give a try (see also instructions below). You will still see “unknown” in the device display, however, no values will be written to DB if no forecast data is available, i.e., you should no longer see “Infinite” values written to DB.

To install from git (assuming installation on Raspbian):

sudo service pimatic stop
cd /home/pi/pimatic-app/node_modules/
sudo rm -rf ./pimatic-openweather
git clone https://github.com/mwittig/pimatic-openweather.git
cd pimatic-openweather
npm install
sudo service pimatic start

To rollback to released version at a later stage

sudo service pimatic stop
cd /home/pi/pimatic-app/node_modules/
sudo rm -rf ./pimatic-openweather
npm install pimatic-openweather
sudo service pimatic start

@mwittig Thank you.
I deleted the database and didn’t receive an invalid value after that. There are no records in the graph now. Before your edit I saw 0 degrees for the min and max in the graph, so I think that works.

@Pazzie Great. Thanks for trying this.

@mwittig said:

The problem for me is (most likely the same for you) is that the forecast data record only contains the data set of today, but the plugin only takes data for the forthcoming days (1-3 days depending on the configuration).

Maybe I’m wrong, but I think you can use weatherLib.daily, instead of weatherLib.forecast (see: http://openweathermap.org/forecast16 and https://www.npmjs.com/package/openweathermap).

I made this with the forecast method (that pimatic-openweather uses):

You are right that it only shows forecast from the same day.

I made this with the daily method:

The unix timestamps (list.dt) convert to:

• Sat, 04 Jul 2015 11:00:00 GMT
• Sun, 05 Jul 2015 11:00:00 GMT
• Mon, 06 Jul 2015 11:00:00 GMT
• Tue, 07 Jul 2015 11:00:00 GMT

The downside is that the values of the daily method are only updated once a day, but that’s fine by me.

EDIT:
You can also increase the cnt value of the forecast method by a lot. It doesn’t get incremented per day, but per three hours. I can use a cnt value of 39, to get all the data for three days:

P.S.
Could you maybe explain me, why you are using:
"if result.list[8*@day]?“
but not something like
"weatherLib.forecastAsync( q: @location, lang: @lang, units: @units, cnt: 8*@day )”

@Pazzie Thanks for pointing this out! I agree, it should be possible to use “weatherLib.daily” which would also simplify the plugin code. Maybe “weatherLib.daily” wasn’t available at the early stages of openweatherlib or there had been some issues with it. Maybe forecast16 is access restricted (requires an API key)?! I’ll discuss that with the other package maintainers. Re your question on the weatherLib.forecast call arguments, I think there is no cnt property. See also REST docs at http://openweathermap.org/forecast5

@mwittig You’re welcome.

Please don’t laugh at my code . I haven’t used Coffeescript before. I just created a quick and dirty fix for myself (https://www.dropbox.com/s/m2xdpdlwswmwlf2/openweather.coffee?dl=0), but you can use it if you wan’t (it probably needs a lot of refactoring ). The weather prediction seems to work, but I’m not 100% sure. I’m using a city ID, instead of a city name (but I don’t know if thats better). If you set the amount of days in the config.js file, then 1 equals today and 2 equals tomorrow etc.

It seems to work without an API key, but I, personally, think its better to add a key.

Maybe I’m wrong, but I think there is a cnt property.
See:

But even if it’s possible to set the cnt property in combination with forecast5, I still think it’s better to use forecast16, since it uses daily average parameters, instead of a random time of the day.

@Pazzie Thanks for sharing your code. Looks great! Using forecast16 is easier than using forecast5. I’ll give it a try later on. See also https://github.com/josecastroleon/pimatic-openweather/issues/16

EDIT: For updates see issue ticket 16

You’re welcome. Thank you for adding the OpenWeatherForecast16Device device! It seems to work, but I haven’t tested it thoroughly.

OpenWeatherMap recommends using a city ID, instead of a city name. Maybe it would be nice to have an optional city ID attribute. For example with an extra attribute. If the city ID attribute isset, then use the city ID and otherwise use the location attirbute. Or with an extra attibute of location type. Where the default setting is name and you can optionally set it to ID.

I also used the arrayday variable, but I think I regret that decision. I think it is easier to just use 0 for today and 1 for tomorrow, just like it was.