Overview

WFL family loggers can export data in many different ways. This topic explains the various options.

First, let’s distinguish between data packets requested (ie pulled) by familiar weather station programs such as Weatherlink for Windows, CumulusMX etc versus the customised export of specific data files.

WFL supports the well-known LOOP and Archive data formats used by Davis stations to pass real-time and summary data packets to a requesting program. Since WFL (and Meteo-Pi Ethernet) are network loggers, this Davis-format packet data passes across the network, usually via port 22222, with WFL effectively emulating the legacy Davis WeatherlinkIP loggers. These direct logger connections are usually made across the local network to software running on a local computer, although in principle these transfers could also be configured to happen across the Internet. This present topic does not cover this type of local data-packet connection to a standard weather station program like CumulusMX – such data connections should just work, with any necessary troubleshooting advice being provided elsewhere in this section – but instead focuses on the other custom export of complete data files that WFL can provide (usually, but not necessarily, to servers across the Internet rather than locally).

Now, let’s make one further distinction: WiFi Logger (WFL) can export (or push) data files to two types of server:

  • Specific weather platforms in the cloud such as weatherlink.com and Weather Underground etc. In this case, data is packaged in the required file format for the target platform;
  • Data packaged in generic file types that can be pointed towards a server of the user’s choice, either local or remote.

For WFL, the specific weather platforms are configured in the Setup | Data Export Options menu, while the generic export options will be managed within the Setup | Custom Export Setup menu. The Meteo-Pi Ethernet (MPE) logger has fewer export options and these are all contained under the Export Setup menu. NB The original Meteo-Pi and DIY loggers are simple interface loggers; these have no user interface and hence no data export options.

Configuration for uploading to specific weather platforms such as weatherlink.com or Weather Underground simply involves entering the appropriate authentication details for each platform and adding any other platform-specific settings. These steps should be fairly obvious to do and again don’t need further comment here.

However, the various options for the generic/custom exports are less obvious and are the main subject of this topic.

Understanding the Custom Data Export options

WFL provides numerous options for using custom data exports and it can be confusing as to which options should be used for which purpose. However, all the alternatives effectively involve the export of a single data file at a given time point and the alternatives are all variations involving three aspects of the file export, as discussed further below:

  • The granularity and frequency of the data within the data file
  • The file contents and file format
  • The protocol being used to upload the data

Data granularity/frequency

As most users will be aware, Davis weather stations can generate two classes of data: current conditions data (also sometimes called real-time data) and summary or archive data.

Current conditions data is a snapshot of sensor readings at a given timepoint and this data sample is regenerated at frequent intervals, as set in the logger configuration. Data files containing current conditions data will contain a single set of readings from all configured sensors and in a format as specified by selected file type. Davis stations can generate new current-conditions readings as often as every 2.5 seconds, but it is rarely necessary or practicable to push data at this frequency across the Internet. It is more common to use a minimum interval of 1 minute or longer, potentially up to a maximum period of 1-2 hours. The frequency is typically configurable within the WFL settings for each upload type.

Archive data is data summarised by the Davis console over a configurable interval from 1 minute to 2 hours, so a single set of readings will contain mean sensor values over the duration of each archive interval. Data files containing archive data will typically contain multiple archive data records covering up to many days of console/logger operation

These two classes of data really serve distinct purposes, with current conditions data being a single snapshot of the latest sensor readings, while the archive data is a more convenient data source to build into a permanent table or database of weather conditions over periods of days, months, years etc, (NB Current conditions data can also be assembled into a full long-term data record at the target server, but using pre-summarised archive data can be a more accurate and more appropriate solution, for reasons that are really beyond the scope of these brief notes.)

File contents and format

TBA

Upload protocol

Protocol in this context means the formal network/Internet protocol by which file transfer is implemented, for example FTP, or MQTT etc. The same file format can be uploaded by different protocols but obviously the sending configuration must match what the target server expects in terms of protocol, port number, authentication credentials etc.

For Custom data exports, WFL currently supports four distinct protocols, as listed below. Note that these are all push protocols in the sense that the logger initiates the data file transfer (at the user’s command in the case of the manual archive transfer, but automatically to a programmable schedule in the case of the other 3 protocols). This is in contrast to the pull requests for LOOP and archive data packets made to the logger by external software, as mentioned in the overview section above. Note that for push protocols, the server must be awake and listening for the data file when it is sent, if data is not to be lost..

Manual archive:

Automatic legacy FTP:

Automatic MQTT

Automatic Unknown::

This topic is still being written.

Last modified: Aug 13, 2023

Feedback

Was this helpful?

Yes No
You indicated this topic was not helpful to you ...
Could you please leave a comment telling us why? Thank you!
Thanks for your feedback.

Post your comment on this topic.

Post Comment