diff --git a/README.md b/README.md index 0d6086d..be8368d 100644 --- a/README.md +++ b/README.md @@ -19,302 +19,41 @@ The production version runs on Amazon Elastic Beanstalk (AWS EB) and therefore t **routes/*.js** contains all the endpoints for the API service. Currently, only two exists for weather adjustment and logging a PWS observation. --- -## Local Installation onto a Raspberry Pi +## Installating a Local Weather Service -**Step 1:** Download and install Node.js onto the Raspberry Pi so that we can run the OpenSprinkler weather server. The version of Node.js to install is dependent on your model of Raspberry Pi. Note that you can run the command ```uname -m``` on your Raspberry Pi to help identify the chipset that is being used. +If you would like to choose between different Weather Providers (currently OpenWeatherMap and DarkSky are supported) or use your local PWS to provide the weather information used by OpenSprinkler then you can install and configure the Weather Service on a device within your own local network. -*For Raspberry Pi 2 or Pi 3 models that are based on the newer ARMv7 and ARMv8 chip* -``` -pi@OSPi:~ $ curl -sL https://deb.nodesource.com/setup_11.x | sudo -E bash -l -pi@OSPi:~ $ sudo apt install -y nodejs -``` +You will need a 24x7 "always on" machine to host the service (this can be a Windows or Linux machine or even a Raspberry Pi device) provided it supports the `Node.js` environment. -*For Raspberry Pi Model A, B, B+, Zero and Compute Module based on the older ARMv6 chip, the process is slightly more convoluted* -``` -pi@OSPi:~ $ wget https://nodejs.org/dist/v11.4.0/node-v11.4.0-linux-armv6l.tar.gz -pi@OSPi:~ $ tar -xvf node-v11.4.0-linux-armv6l.tar.gz -pi@OSPi:~ $ cd node-v11.4.0-linux-armv6l -pi@OSPi:~ $ sudo cp -R * /usr/local/ -pi@OSPi:~ $ cd .. -pi@OSPi:~ $ rm -rf node-v11.4.0-linux-armv6l -pi@OSPi:~ $ rm node-v11.4.0-linux-armv6l.tar.gz - -``` - -**Step 2:** Download the OpenSprinkler Weather Service repository to your Raspberry Pi so that we can run a local version of the service: - -``` -pi@OSPi:~ $ git clone https://github.com/OpenSprinkler/OpenSprinkler-Weather.git weather -``` - -**Step 3:** Install all of the dependencies using the Node Package Manager, npm, from within the weather project directory and transpile the TypeScript files to JavaScript: -``` -pi@OSPi:~ $ cd weather -pi@OSPi:~/weather $ npm install -pi@OSPi:~/weather $ npm run compile -``` -**Step 4:** Configure the weather server to use either the OpenWeatherMap API or the Dark Sky API - -* **Step 4a:** If you want to use the Open Weather Map API, go to `https://openweathermap.org/appid` to register with OpenWeatherMaps and obtain an API key that is needed to request weather information. - -* **Step 4b:** If you want to use the Dark Sky API, go to `https://darksky.net/dev` to register with Dark Sky and obtain an API key that is needed to request weather information. - -**Step 5:** The file .env is used by the weather server to specify the interface and port to listen on for OpenSprinkler Firmware weather requests. We need to create a new file, .env, and enter some configuration details. -``` -pi@OSPi:~/weather $ nano .env -``` - -Add the following two lines to the .env file so that the weather server is configured to listen for weather requests. Using 0.0.0.0 as the host interfaces allows you to access the service from another machine to test. Alternatively, set HOST to “localhost” if you want to limit weather service access to only applications running on the local machine. - -``` -HOST=0.0.0.0 -PORT=3000 -``` - -If you want to use the OWM API, also add the following two lines to the .env file: -``` -WEATHER_PROVIDER=OWM -OWM_API_KEY= -``` - -If you want to use the Dark Sky API instead, add these two lines to the .env file: -``` -WEATHER_PROVIDER=DarkSky -DARKSKY_API_KEY= -``` - - -**Step 6:** Setup the Weather Server to start whenever the Raspberry Pi boots up using the built-in service manager: - -``` -pi@OSPi:~/weather $ sudo nano /etc/systemd/system/weather.service -``` - -Cut and paste the following lines into the weather.service file: - -``` -[Unit] -Description=OpenSprinkler Weather Server - -[Service] -ExecStart=/usr/bin/npm start -WorkingDirectory=/home/pi/weather -Restart=always -RestartSec=10 - -[Install] -WantedBy=multi-user.target -``` -Save the file and enable/start the weather service - -``` -pi@OSPi:~/weather $ sudo systemctl enable weather.service -pi@OSPi:~/weather $ sudo systemctl start weather.service -pi@OSPi:~/weather $ systemctl status weather.service -``` - -The final line above checks that the service has been started and you should see the service marked as running. - -**Step 7:** You can now test that the service is running correctly from a Web Browser by navigating to the service (note: the default port was set to 3000 in the .env file): - -``` -http:/// -``` -You should see "OpenSprinkler Weather Service" in response. - -You can use the following request to see the watering level that the Weather Service calculates. Note: to be consistent, change the values of h, t and r to the % weightings and bh (as a %), bt (in F), bp (in inches) to the offsets from the Zimmerman config page in App. - -``` -http://:3000/weather1.py?loc=50,1&wto="h":100,"t":100,"r":100,"bh":70,"bt":59,"br":0 -``` - -This will return a response similar to below with ```scale``` value equating to the watering level and ```rawData``` reflecting the temp (F), humidity (%) and daily rainfall (inches) used in the zimmerman calc. -``` -&scale=20&rd=-1&tz=48&sunrise=268&sunset=1167&eip=3232235787&rawData={"h":47,"p":0,"t":54.4,"raining":0} -``` - -**Step 8:** You will now need to configure your OpenSprinkler device to use the local version of the Weather Service rather than the Cloud version. - -- For OS users, go to `http:///su` on your web browser - -- For OSPi users, go to `http://:8080/su` on your web browser - -In the **Weather** text field, specify the IP of your Weather Service server and Port, e.g. for OSPi users this might be “localhost:3000” if the Weather Service is running on the same Raspberry Pi. - -OpenSprinkler should now be connected to your local Weather Service for calculating rain delay and watering levels. +For detailed instructions on setup and configuration of a local Weather Service running on a Raspberry Pi then click [here](docs/local-installation.md) --- -## Submitting PWS Observations to a Local Weather Service Server +## Connecting a Personal Weather Station to a Local Weather Service + +If you are running a local instance of the Weather Service then you may be able to send the data directly from your PWS to the Weather Service avoiding any "cloud" based services. The weather data can then be used by the Weather Service to calculate Zimmerman based watering levels. ### Options for PWS Owners -**1 ) PWS Supporting RESTfull Output** +**1 ) PWS supporting RESTfull output** -Some PWS allow the user to specify a `GET request` to send weather observations onto a local service for processing. For example, the MeteoBridge Pro allows for requests to be specified in a custom template that translates the PWS weather values and units into a format that the local OS Weather Service can accept. If available, The user documentation for the PWS should detail how to configure a custom GET request and the message structure required by the OS Weather Service is documented below. +Some PWS allow the user to specify a `GET request` to send weather observations onto a local service for processing. For example, the MeteoBridge Pro allows for requests to be specified in a custom template that translates the PWS weather values and units into a format that the local Weather Service can accept. If available, the user documentation for the PWS should detail how to configure a custom GET request. -**2 ) Networked PWS Supporting Weather Underground** +For more information on the RESTfull protocol click [here](docs/pws-protocol.md) -Many PWS already support the Weather Underground format and can be connected to the user's home network to send data directly to the WU cloud service. For these PWS, it is possible to physically intercept the data stream heading to the WU cloud and redirect it to the OS Weather Service server instead. +**2 ) Networked PWS that support Weather Underground** + +Many PWS already support the Weather Underground format and can be connected to the user's home network to send data directly to the WU cloud service. For these PWS, it is possible to physically intercept the data stream heading to the WU cloud and redirect it to the Weather Service server instead. To do this intercepting, you place a physical device - such as a Raspberry Pi - in-between the PWS and the home network. It is this "man-in-the-middle" device that will look for information heading from the PWS toward the WU cloud and redirect that information to the local Weather Service. -There is a section below that goes through the steps to configure/install a Raspberry Pi Zero W to act as a "man-in-the-middle". +For more information on configuring a Raspberry Pi Zero W to act as a "Man In The Middle" solution click [here](docs/man-in-middle.md) **3 ) PWS Supported By WeeWX** -From the Author of [WeeWX](http://www.weewx.com) - *"WeeWX is a free, open source, software program, written in Python, which interacts with your weather station to produce graphs, reports, and HTML pages. It can optionally publish to weather sites or web servers. It uses modern software concepts, making it simple, robust, and easy to extend. It includes extensive documentation."* +The WeeWX project provides a mechanism for OpenSprinkler owners to capture the data from many different manufacturer's PWS and to both store the information locally and to publish the data to a number of destinations. OpenSprinkler owners can use this solution to send their PWS weather observations onto a local Weather Service server. -The WeeWX project provides a mechanism for OpenSprinkler PWS Owners to capture the data from their PWS and to both store the information locally and to publish the data to a number of destinations. OpenSprinkler Users can use this solution to send their PWS weather observations onto a local Weather Service server. +For more information on the "WeeWX Solution" click [here](docs/weewx.md) -The list of WeeWX supported hardware can be found [here](http://www.weewx.com/hardware.html) along with an extensive installation/configuration documentation [here](http://www.weewx.com/docs.html). There is also an active Google Groups forum [here](https://groups.google.com/forum/#!forum/weewx-user) +**4 ) Solutions for Other PWS** -Once installed and capturing data, the WeeWX solution can send the weather observation onto the local Weather Service. WeeWX's built-in Weather Underground plug-in can be configured in the ```/etc/weewx/weewx.conf``` file specifying the IP Address and Port of the local Weather Service server as follows: - -``` - [[Wunderground]] - enable = true - station = anyText - password = anyText - server_url = http://:/weatherstation/updateweatherstation.php - rapidfire = False -``` -Note: the `station` and `password` entries are not used by the OS Weather Service but must be populated to keep the plug-in happy. - -You can then restart the WeeWX server and your PWS observations should now be sent to the local Weather Service every 5 minutes. - - ---- -## Personal Weather Station Upload Protocol - -**Background** - -To upload a PWS observation, you make a standard HTTP GET request with the weather conditions as the GET parameters. - -**Endpoint** - -The GET message should be directed to the local Weather Service server and with the same endpoint as used by legacy Weather Underground service: - -``` -http:///weatherstation/updateweatherstation.php -``` - -**GET Parameters** - -The following fields are required: - - -| Field Name | Format | Description | -|---|:---:|---| -| tempf | 55\.6 | Outdoor temperature in fahrenheit | -| humidity | 0-100 | Outdoor humidity as a percentage | -| rainin | 0.34 | Accumulated rainfall in inches over the last 60 min | -| dailyrainin | 1.45 | Accumulated rainfall in inches for the current day (in local time) | -| dateutc | 2019-03-12 07:45:10 | Time in UTC as YYYY-MM-DD HH:MM:SS (not local time) | - -IMPORTANT all fields must be url escaped. For example, if the current time in utc is "`2019-01-01 10:32:35`" then the dateutc field should be sent as "`2019-01-01+10%3A32%3A35`". For reference see http://www.w3schools.com/tags/ref_urlencode.asp. - -_[To Do: If the weather station is not capable of producing a timestamp then either omit the field or set the field value to "`now`"]_ - - -**Example GET Message** - -Here is an example of a full URL: -``` -https:///weatherstation/updateweatherstation.php?tempf=70.5&humidity=90&rainin=0&dailytainin=0.54&dateutc=2000-01-01+10%3A32%3A35 -``` -The response text from the Weather Service server will be either "`success`" or an error message. - ---- -## Setup a Raspberry Pi To Intercept PWS Information - -The following steps are based on a Raspberry Pi Zero W with an Ethernet/USB adapter to provide two network interfaces. The installation instructions below assume the PWS Internet Bridge has been connected into the Pi's ethernet port and that the Pi's WiFi interface is being used to connect with the Home Network. - -**Step 1: Install Software and Basic Setup** - -Install the latest version of Raspbian onto the Pi and configure the wifi network as per the instructions from the Raspberry Pi Foundation. You can now `ssh` into the Pi via the WiFi network and contiue the setup process. - -We only need to install one additional piece of software called `dnsmasq` which we will need to manage the network on the ethernet side of the Pi. We don't want any of the default configuration as we need to tailor that to our specific needs: - -``` -pi@raspberry:~ $ sudo apt-get install dnsmasq -pi@raspberry:~ $ sudo rm -rf /etc/dnsmasq.d/* -``` - -Lastly, we need to change one of the default Raspberry Pi setting to enable IP forwarding. We will be using this forwarding functional later in the installation process. The setting can be changed by editing the file `sysctl.conf`: - -``` -pi@raspberry:~ $ sudo nano /etc/sysctl.conf -``` -Uncomment the line "`# net.ipv4.ip_forward=1`" to look as follows and save the file: -``` -net.ipv4.ip_forward=1 -``` -We now have a pretty standard Raspberry Pi installation with the Pi connected to our Home Network via the WiFi interface. - -**Step 2: Configure the PWS Side of the Network** - -We now need to shift our focus across to the ethernet side of the Pi. At the moment, we have the PWS physically connected to the Pi via the ethernet port but have yet to setup the networking layer to communicate with the PWS. - -Next, we to assign a static address to the Pi's ethernet port (`eth0`). This is the port connected to the PWS Internet Bridge and will act as the "network controller" for the ethernet side of things. Since my home network is configured to use `192.168.1.0-255`, I choose to use `192.168.2.0-255` for the network on the ethernet side. The commands below setup a network using this address range. So we need to edit the `dhcp.conf` configuration file - -``` -pi@raspberry:~ $ sudo nano /etc/dhcpcd.conf -``` - -Adding the following lines to the end of the file: - -``` -interface eth0 -static ip_address=192.168.2.1/24 -static routers=192.168.2.0 -``` - -Now we need to configure `dnsmasq` to allocate an IP address to our PWS Internet Gateway so that it can connect and communicate with the Pi. In order for the PWS to get the same static address each time it restarts, we need to tell `dnsmasq` the MAC address of the PWS and the Hostname and IP Address we wantit to have. For example, my Ambient Weather PWS has a MAC Address of 00:0E:C6:XX:XX:XX and I want it to be known as "PWS" at 192.168.2.10. - -We need to create a new file to configure our specific requirements: -``` -pi@raspberry:~ $ sudo nano /etc/dnsmasq.d/eth0-dnsmasq.conf -``` -Add the following lines of configuration to the file (swapping out , and with our required values): -``` -interface=eth0 -bind-interfaces -server=8.8.8.8 -domain-needed -bogus-priv -dhcp-range=192.168.2.2,192.168.2.100,12h -dhcp-host=,, -``` -**Step 3: Configure the Intercept (Port Forwarding)** - -Now that we have both sides of the network configured, we can setup the Pi to intercept weather observations sent by the PWS Internet Bridge to Weather Underground. We do this by identifying all packets arriving at the Pi from the PWS Internet Gateway and heading towards Port 80 (the WU cloud port). - -These packets can be redirected to the IP and Port of our local Weather Service using the `iptable` command. We will need to setup the configuration and then save it to a file `iptables.ipv4.nat` so that we can restore the configuration easily after a reboot. When executing the commands below, make sure to substitute with the PWS address selected earlier and to use the IP and Port for your local Weather Service in place of : - -``` -pi@raspberry:~ $ sudo iptables -t nat -A PREROUTING -s -p tcp --dport 80 -j DNAT --to-destination -pi@raspberry:~ $ sudo iptables -t nat -A POSTROUTING -j MASQUERADE -pi@raspberry:~ $ sudo iptables-save > /etc/iptables.ipv4.nat -``` -In order to ensure these forwarding rules are always operating, we need to create a small batch file called `/etc/network/if-up.d/eth0-iptables` that is run every time the ethernet inerface is started: -``` -pi@raspberry:~ $ sudo nano /etc/network/if-up.d/eth0-iptables -``` -Add the following lines: -``` -#!/bin/sh -sudo iptables-restore < /etc/iptables.ipv4.nat -``` -Lastly, ensure that the file is executable: -``` -pi@raspberry:~ $ chmod +x /etc/network/if-up.d/eth0-iptables -``` -We have now configured the various port forwarding rules and ensured they will survive a reboot and/or a restart of the ethernet interface. - -**Step 4: Start the Redirection of Weather Observations** - -All of the configuration has been completed and the Raspberry Pi can be rebooted to activate the redirection of PWS observations to the local Weather Service: - -``` -pi@raspberry:~ $ sudo reboot -``` +- Davis Vantage: a solution for this PWS has been kindly provided by @rmloeb [here](docs/davis-vantage.md) \ No newline at end of file diff --git a/docs/davis-vantage.md b/docs/davis-vantage.md new file mode 100644 index 0000000..da85d48 --- /dev/null +++ b/docs/davis-vantage.md @@ -0,0 +1,28 @@ +## Connecting a Davis Vantage PWS to the Local Weather Service + +**Background** + +The Davis Vantage has the option to connect the PWS to a Windows machine using a serial or USB logger. The Davis Weatherlink software has an optional DLL, WUiWlink_1.dll (dated 4/26/17), which is designed to push observation data from the Davis Vantage weather console to WeatherUnderground's now obsolete interface. That feature can be used to push data to a local instance of weather-service, which will, in turn, be accessed by OpenSprinkler as data for the Zimmerman water level calculation. + +Note: if you have the WeatherLinkIP version then see the instructions for using a RaspberryPi Zero to redirect weather data to local weather-service. + +**Configuration** + +To install the DLL module, see the directions at http://www.davisinstruments.com/resource/send-weather-data-weather-underground-weatherlink-software/ + +To redirect the weather data to weather-server, modify the HOSTS file on the Windows machine running Weatherlink by adding the following two lines substituting `` for the IP address of your local Weather Service: +``` +local rtupdate.wunderground.com +local weatherstation.wunderground.com +``` +Note: you must be running in administrator mode to make this change. On Windows 10 the HOSTS file is in `C:/Windows/System32/drivers/etc`. The easiest way to do this is to open a Command Prompt in Admin mode, navigate to `C:/Windows/System32/drivers/etc`, then execute "notepad.exe hosts", add the three entries, save, exit, and close the command window. The change should take effect immediately, but you may need to reboot the Windows machine to be sure. + +In the Weatherlink application you should see "Wunderground settings" in the File menu. You can ignore the StationID and Password settings (or just enter a single blank character). Set the Update Interval to 5 minutes, which should be more than sufficient for the purpose of the Zimmerman water level calculation. + +On the machine running weather-server, edit the weather-server `.env` file to add a line `"PWS=WU"`. Stop and restart weather-service. + +Actual readings from your PWS should now be flowing to weather-service. Make sure you have Zimmerman selected in OpenSprinkler and set the parameters appropriately for your situation. + +**Testing** + +To immediately observe the data feed, open Davis WeatherLink, click on File | Wunderground Settings, then click the "Test" box. \ No newline at end of file diff --git a/docs/local-installation.md b/docs/local-installation.md new file mode 100644 index 0000000..a089999 --- /dev/null +++ b/docs/local-installation.md @@ -0,0 +1,120 @@ +## Installating a Local Weather Service onto a Raspberry Pi + +**Step 1:** Download and install Node.js onto the Raspberry Pi so that you can run the OpenSprinkler weather server locally. The version of Node.js to install is dependent on your model of Raspberry Pi. Note that you can run the command ```uname -m``` on your Raspberry Pi to help identify the chipset that is being used. + +*For Raspberry Pi 2 or Pi 3 models that are based on the newer ARMv7 and ARMv8 chip* +``` +pi@OSPi:~ $ curl -sL https://deb.nodesource.com/setup_11.x | sudo -E bash -l +pi@OSPi:~ $ sudo apt install -y nodejs +``` + +*For Raspberry Pi Model A, B, B+, Zero and Compute Module based on the older ARMv6 chip, the process is slightly more convoluted* +``` +pi@OSPi:~ $ wget https://nodejs.org/dist/v11.4.0/node-v11.4.0-linux-armv6l.tar.gz +pi@OSPi:~ $ tar -xvf node-v11.4.0-linux-armv6l.tar.gz +pi@OSPi:~ $ cd node-v11.4.0-linux-armv6l +pi@OSPi:~ $ sudo cp -R * /usr/local/ +pi@OSPi:~ $ cd .. +pi@OSPi:~ $ rm -rf node-v11.4.0-linux-armv6l +pi@OSPi:~ $ rm node-v11.4.0-linux-armv6l.tar.gz + +``` + +**Step 2:** Download the OpenSprinkler Weather Service repository to your Raspberry Pi so that you can run a local version of the service: + +``` +pi@OSPi:~ $ git clone https://github.com/OpenSprinkler/OpenSprinkler-Weather.git weather +``` + +**Step 3:** Install all of the dependencies using the Node Package Manager, `npm`, from within the weather project directory and transpile the TypeScript files to JavaScript: +``` +pi@OSPi:~ $ cd weather +pi@OSPi:~/weather $ npm install +pi@OSPi:~/weather $ npm run compile +``` +**Step 4:** Configure the weather server to use either the OpenWeatherMap API or the Dark Sky API + +* **Step 4a:** If you want to use the Open Weather Map API, go to `https://openweathermap.org/appid` to register with OpenWeatherMaps and obtain an API key that is needed to request weather information. + +* **Step 4b:** If you want to use the Dark Sky API, go to `https://darksky.net/dev` to register with Dark Sky and obtain an API key that is needed to request weather information. + +**Step 5:** The file `.env` is used by the weather server to specify the interface and port to listen on for requests coming from your OpenSprinkler device. You need to create a new `.env` file and enter some configuration details. +``` +pi@OSPi:~/weather $ nano .env +``` + +Add the following two lines to the .env file so that the weather server is configured to listen for weather requests. Using 0.0.0.0 as the host interfaces allows you to access the service from another machine to test. Alternatively, set HOST to “localhost” if you want to limit weather service access to only applications running on the local machine. + +Note: if you are using OS then you must set `PORT=80` as this cannot be changed on the OS device. If using OSPi or OSBo then you can set `PORT` to any unused value. + +``` +HOST=0.0.0.0 +PORT=3000 +``` + +If you want to use the OWM API, also add the following two lines to the .env file: +``` +WEATHER_PROVIDER=OWM +OWM_API_KEY= +``` + +If you want to use the Dark Sky API instead, add these two lines to the .env file: +``` +WEATHER_PROVIDER=DarkSky +DARKSKY_API_KEY= +``` + + +**Step 6:** Setup the Weather Server to start whenever the Raspberry Pi boots up using the built-in service manager: + +``` +pi@OSPi:~/weather $ sudo nano /etc/systemd/system/weather.service +``` + +Cut and paste the following lines into the weather.service file: + +``` +[Unit] +Description=OpenSprinkler Weather Server + +[Service] +ExecStart=/usr/bin/npm start +WorkingDirectory=/home/pi/weather +Restart=always +RestartSec=10 + +[Install] +WantedBy=multi-user.target +``` +Save the file and enable/start the weather service + +``` +pi@OSPi:~/weather $ sudo systemctl enable weather.service +pi@OSPi:~/weather $ sudo systemctl start weather.service +pi@OSPi:~/weather $ systemctl status weather.service +``` + +The final line above checks that the service has been started and you should see the service marked as running. + +**Step 7:** You can now test that the service is running correctly from a Web Browser by navigating to the service (note: make sure to use the PORT number specified in the `.env` file above, e.g. 3000 for OSPi or 80 for OS devices): + +``` +http:/// +``` +You should see the text "OpenSprinkler Weather Service" appear in your browser in response. + +Next, you can use the following request to see the watering level that the Weather Service calculates. Note: to be consistent, change the values of h, t and r to the % weightings and bh (as a %), bt (in F), bp (in inches) to the offsets from the Zimmerman config page in App. + +``` +http:///weather1.py?loc=50,1&wto="h":100,"t":100,"r":100,"bh":70,"bt":59,"br":0 +``` + +This will return a response similar to below with the `scale` value equating to the watering level and `rawData` reflecting the temp (F), humidity (%) and daily rainfall (inches) used in the zimmerman calc. +``` +&scale=20&rd=-1&tz=48&sunrise=268&sunset=1167&eip=3232235787&rawData={"h":47,"p":0,"t":54.4,"raining":0} +``` + +**Step 8:** You will now need to configure your OpenSprinkler device to use the local version of the Weather Service rather than the Cloud version. On a web browser, go to `http://:80/su` if you have an OS device or `http://:8080/su` for OSPi/OSBo devices to set the Weather Service IP and PORT number. + +OpenSprinkler should now be connected to your local Weather Service for calculating rain delay and watering levels. + diff --git a/docs/man-in-middle.md b/docs/man-in-middle.md new file mode 100644 index 0000000..6eed125 --- /dev/null +++ b/docs/man-in-middle.md @@ -0,0 +1,92 @@ +## Setup a Raspberry Pi To Intercept PWS Information + +The following steps are based on a Raspberry Pi Zero W with an Ethernet/USB adapter to provide two network interfaces. The installation instructions below assume the PWS Internet Bridge has been connected into the Pi's ethernet port and that the Pi's WiFi interface is being used to connect with the Home Network. + +**Step 1: Install Software and Basic Setup** + +Install the latest version of Raspbian onto the Pi and configure the wifi network as per the instructions from the Raspberry Pi Foundation. You can now `ssh` into the Pi via the WiFi network and contiue the setup process. + +We only need to install one additional piece of software called `dnsmasq` which we will need to manage the network on the ethernet side of the Pi. We don't want any of the default configuration as we need to tailor that to our specific needs: + +``` +pi@raspberry:~ $ sudo apt-get install dnsmasq +pi@raspberry:~ $ sudo rm -rf /etc/dnsmasq.d/* +``` + +Then, we need to change one of the default Raspberry Pi setting to enable IP forwarding. We will be using this forwarding functional later in the installation process. The setting can be changed by editing the file `sysctl.conf`: + +``` +pi@raspberry:~ $ sudo nano /etc/sysctl.conf +``` +Uncomment the line "`# net.ipv4.ip_forward=1`" to look as follows and save the file: +``` +net.ipv4.ip_forward=1 +``` +We now have a pretty standard Raspberry Pi installation with the Pi connected to our Home Network via the WiFi interface. + +**Step 2: Configure the PWS Side of the Network** + +We now need to shift our focus across to the ethernet side of the Pi. At the moment, we have the PWS physically connected to the Pi via the ethernet port but have yet to setup the networking layer to communicate with the PWS. + +Wwe need to assign a static address to the Pi's ethernet port (`eth0`). This is the port connected to the PWS Internet Bridge and will act as the "network controller" for the ethernet side of things. Since my home network is configured to use `192.168.1.0-255`, I choose to use `192.168.2.0-255` for the network on the ethernet side. To make these changes, we need to edit the `dhcp.conf` configuration file: + +``` +pi@raspberry:~ $ sudo nano /etc/dhcpcd.conf +``` + +Adding the following lines to the end of the file: + +``` +interface eth0 +static ip_address=192.168.2.1/24 +static routers=192.168.2.0 +``` + +Now we need to configure `dnsmasq` to allocate an IP address to our PWS Internet Gateway so that it can connect and communicate with the Pi. In order for the PWS to get the same static address each time it restarts, we need to tell `dnsmasq` the MAC address of the PWS and the Hostname and IP Address we want it to have. For example, my Ambient Weather PWS has a MAC Address of 00:0E:C6:XX:XX:XX and I want it to be known as "PWS" at 192.168.2.10. + +We need to create a new file to configure our specific requirements: +``` +pi@raspberry:~ $ sudo nano /etc/dnsmasq.d/eth0-dnsmasq.conf +``` +Add the following lines of configuration to the file (swapping out , and with our required values): +``` +interface=eth0 +bind-interfaces +server=8.8.8.8 +domain-needed +bogus-priv +dhcp-range=192.168.2.2,192.168.2.100,12h +dhcp-host=,, +``` +**Step 3: Configure the Intercept (Port Forwarding)** + +Now that we have both sides of the network configured, we can setup the Pi to intercept weather observations sent by the PWS Internet Bridge to Weather Underground. We do this by identifying all packets arriving at the Pi from the PWS Internet Gateway and heading towards Port 80 (the WU cloud port). + +These packets can be redirected to the IP and Port of our local Weather Service using the `iptable` command. We will need to setup the configuration and then save it to a file `iptables.ipv4.nat` so that we can restore the configuration easily after a reboot. When executing the commands below, make sure to substitute with the PWS address selected earlier and to use the IP and Port for your local Weather Service in place of ``: +``` +pi@raspberry:~ $ sudo iptables -t nat -A PREROUTING -s -p tcp --dport 80 -j DNAT --to-destination +pi@raspberry:~ $ sudo iptables -t nat -A POSTROUTING -j MASQUERADE +pi@raspberry:~ $ sudo iptables-save > /etc/iptables.ipv4.nat +``` +In order to ensure these forwarding rules are always operating, we need to create a small batch file called `/etc/network/if-up.d/eth0-iptables` that is run every time the ethernet inerface is started: +``` +pi@raspberry:~ $ sudo nano /etc/network/if-up.d/eth0-iptables +``` +Add the following lines: +``` +#!/bin/sh +sudo iptables-restore < /etc/iptables.ipv4.nat +``` +Lastly, ensure that the file is executable: +``` +pi@raspberry:~ $ chmod +x /etc/network/if-up.d/eth0-iptables +``` +We have now configured the various port forwarding rules and ensured they will survive a reboot and/or a restart of the ethernet interface. + +**Step 4: Start the Redirection of Weather Observations** + +All of the configuration has been completed and the Raspberry Pi can be rebooted to activate the redirection of PWS observations to the local Weather Service: + +``` +pi@raspberry:~ $ sudo reboot +``` diff --git a/docs/pws-protocol.md b/docs/pws-protocol.md new file mode 100644 index 0000000..ab1ba34 --- /dev/null +++ b/docs/pws-protocol.md @@ -0,0 +1,40 @@ +## Personal Weather Station Upload Protocol + +**Background** + +To upload a PWS observation, you make a standard HTTP GET request with the weather conditions as the GET parameters. + +**Endpoint** + +The GET message should be directed to the local Weather Service server and with the same endpoint as used by legacy Weather Underground service: + +``` +http:///weatherstation/updateweatherstation.php +``` + +**GET Parameters** + +The following fields are required: + + +| Field Name | Format | Description | +|---|:---:|---| +| tempf | 55\.6 | Outdoor temperature in fahrenheit | +| humidity | 0-100 | Outdoor humidity as a percentage | +| rainin | 0.34 | Accumulated rainfall in inches over the last 60 min | +| dailyrainin | 1.45 | Accumulated rainfall in inches for the current day (in local time) | +| dateutc | 2019-03-12 07:45:10 | Time in UTC as YYYY-MM-DD HH:MM:SS (not local time) | + +IMPORTANT all fields must be url escaped. For example, if the current time in utc is "`2019-01-01 10:32:35`" then the dateutc field should be sent as "`2019-01-01+10%3A32%3A35`". For reference see http://www.w3schools.com/tags/ref_urlencode.asp. + +_[To Do: If the weather station is not capable of producing a timestamp then either omit the field or set the field value to "`now`"]_ + + +**Example GET Message** + +Here is an example of a full URL: +``` +https:///weatherstation/updateweatherstation.php?tempf=70.5&humidity=90&rainin=0&dailytainin=0.54&dateutc=2000-01-01+10%3A32%3A35 +``` +The response text from the Weather Service server will be either "`success`" or an error message. + diff --git a/docs/weewx.md b/docs/weewx.md new file mode 100644 index 0000000..2ccb303 --- /dev/null +++ b/docs/weewx.md @@ -0,0 +1,25 @@ +## The WeeWX Project + +**Background** + +From the Author of [WeeWX](http://www.weewx.com) - *"WeeWX is a free, open source, software program, written in Python, which interacts with your weather station to produce graphs, reports, and HTML pages. It can optionally publish to weather sites or web servers. It uses modern software concepts, making it simple, robust, and easy to extend. It includes extensive documentation."* + +**Supported Weather Stations** + +The list of WeeWX supported hardware can be found [here](http://www.weewx.com/hardware.html) along with an extensive installation/configuration documentation [here](http://www.weewx.com/docs.html). There is also an active Google Groups forum [here](https://groups.google.com/forum/#!forum/weewx-user) + +**Connecting WeeWX to OpenSprinkler Weather Service** + +Once installed and capturing data, the WeeWX solution can send the weather observation onto the local Weather Service. WeeWX's built-in Weather Underground plug-in can be configured in the ```/etc/weewx/weewx.conf``` file specifying the IP Address and Port of the local Weather Service server as follows: + +``` + [[Wunderground]] + enable = true + station = anyText + password = anyText + server_url = http://:/weatherstation/updateweatherstation.php + rapidfire = False +``` +Note: the `station` and `password` entries are not used by the OS Weather Service but must be populated to keep the plug-in happy. + +You can then restart the WeeWX server and your PWS observations should now be sent to the local Weather Service every 5 minutes. diff --git a/routes/weather.ts b/routes/weather.ts index be4f644..dc4c053 100644 --- a/routes/weather.ts +++ b/routes/weather.ts @@ -221,6 +221,7 @@ export const getWateringData = async function( req: express.Request, res: expres // parsed. This allows the adjustment method and the restriction type to both // be saved in the same byte. let adjustmentMethod: number = req.params[ 0 ] & ~( 1 << 7 ), + checkRestrictions: boolean = ( ( req.params[ 0 ] >> 7 ) & 1 ) > 0, adjustmentOptionsString: string = getParameter(req.query.wto), location: string | GeoCoordinates = getParameter(req.query.loc), outputFormat: string = getParameter(req.query.format), @@ -262,7 +263,7 @@ export const getWateringData = async function( req: express.Request, res: expres let wateringData: WateringData; if ( local.useLocalWeather() ) { wateringData = await getLocalWateringData( coordinates ); - } else if ( adjustmentMethod !== 0 ) { + } else if ( adjustmentMethod !== 0 || checkRestrictions ) { wateringData = await weatherProvider.getWateringData(coordinates); } diff --git a/routes/weatherProviders/DarkSky.ts b/routes/weatherProviders/DarkSky.ts index c934d94..6153dd5 100644 --- a/routes/weatherProviders/DarkSky.ts +++ b/routes/weatherProviders/DarkSky.ts @@ -5,37 +5,71 @@ import { httpJSONRequest } from "../weather"; async function getDarkSkyWateringData( coordinates: GeoCoordinates ): Promise< WateringData > { // The Unix timestamp of 24 hours ago. - const timestamp: number = moment().subtract( 1, "day" ).unix(); + const yesterdayTimestamp: number = moment().subtract( 1, "day" ).unix(); + const todayTimestamp: number = moment().unix(); const DARKSKY_API_KEY = process.env.DARKSKY_API_KEY, - historicUrl = `https://api.darksky.net/forecast/${DARKSKY_API_KEY}/${coordinates[0]},${coordinates[1]},${timestamp}`; + yesterdayUrl = `https://api.darksky.net/forecast/${DARKSKY_API_KEY}/${coordinates[0]},${coordinates[1]},${yesterdayTimestamp}?exclude=currently,minutely,daily,alerts,flags`, + todayUrl = `https://api.darksky.net/forecast/${DARKSKY_API_KEY}/${coordinates[0]},${coordinates[1]},${todayTimestamp}?exclude=currently,minutely,daily,alerts,flags`; - let historicData; + let yesterdayData, todayData; try { - historicData = await httpJSONRequest( historicUrl ); + yesterdayData = await httpJSONRequest( yesterdayUrl ); + todayData = await httpJSONRequest( todayUrl ); } catch (err) { // Indicate watering data could not be retrieved if an API error occurs. return undefined; } + if ( !todayData.hourly || !todayData.hourly.data || !yesterdayData.hourly || !yesterdayData.hourly.data ) { + return undefined; + } + + /* The number of hourly forecasts to use from today's data. This will only include elements that contain historic + data (not forecast data). */ + // Find the first element that contains forecast data. + const todayElements = Math.min( 24, todayData.hourly.data.findIndex( ( data ) => data.time > todayTimestamp - 60 * 60 ) ); + + /* Take as much data as possible from the first elements of today's data and take the remaining required data from + the remaining data from the last elements of yesterday's data. */ + const samples = [ + ...yesterdayData.hourly.data.slice( todayElements - 24 ), + ...todayData.hourly.data.slice( 0, todayElements ) + ]; + + // Fail if not enough data is available. + if ( samples.length !== 24 ) { + return undefined; + } + + const totals = { temp: 0, humidity: 0, precip: 0 }; + for ( const sample of samples ) { + totals.temp += sample.temperature; + totals.humidity += sample.humidity; + totals.precip += sample.precipIntensity + } + return { - // Calculate average temperature for the day using hourly data. - temp : historicData.hourly.data.reduce( ( sum, hourlyData ) => sum + hourlyData.temperature, 0 ) / historicData.hourly.data.length, - humidity: historicData.daily.data[ 0 ].humidity * 100, - precip: historicData.daily.data[ 0 ].precipIntensity * 24, - raining: historicData.currently.precipType === "rain" + temp : totals.temp / 24, + humidity: totals.humidity / 24 * 100, + precip: totals.precip, + raining: samples[ samples.length - 1 ].precipIntensity > 0 }; } async function getDarkSkyWeatherData( coordinates: GeoCoordinates ): Promise< WeatherData > { const DARKSKY_API_KEY = process.env.DARKSKY_API_KEY, - forecastUrl = `https://api.darksky.net/forecast/${DARKSKY_API_KEY}/${coordinates[0]},${coordinates[1]}`; + forecastUrl = `https://api.darksky.net/forecast/${DARKSKY_API_KEY}/${coordinates[0]},${coordinates[1]}?exclude=minutely,alerts,flags`; let forecast; try { forecast = await httpJSONRequest( forecastUrl ); } catch (err) { - // Indicate watering data could not be retrieved if an API error occurs. + // Indicate weather data could not be retrieved if an API error occurs. + return undefined; + } + + if ( !forecast.currently || !forecast.daily || !forecast.daily.data ) { return undefined; } @@ -49,16 +83,16 @@ async function getDarkSkyWeatherData( coordinates: GeoCoordinates ): Promise< We region: "", city: "", - minTemp: Math.floor( forecast.daily.data[ 0 ].temperatureLow ), - maxTemp: Math.floor( forecast.daily.data[ 0 ].temperatureHigh ), + minTemp: Math.floor( forecast.daily.data[ 0 ].temperatureMin ), + maxTemp: Math.floor( forecast.daily.data[ 0 ].temperatureMax ), precip: forecast.daily.data[ 0 ].precipIntensity * 24, - forecast: [] + forecast: [ ] }; for ( let index = 0; index < forecast.daily.data.length; index++ ) { weather.forecast.push( { - temp_min: Math.floor( forecast.daily.data[ index ].temperatureLow ), - temp_max: Math.floor( forecast.daily.data[ index ].temperatureHigh ), + temp_min: Math.floor( forecast.daily.data[ index ].temperatureMin ), + temp_max: Math.floor( forecast.daily.data[ index ].temperatureMax ), date: forecast.daily.data[ index ].time, // TODO set this icon: "",