Overview
This topic introduces the v2 API for accessing data from weatherlink.com. The previous v1 API is covered in a separate topic. Only weather stations made by Davis Instruments are permitted to upload to weatherlink.com.
Be aware that the v2 API has been completely rewritten from the v1 version with different objectives and different data formats & methods; v1 and v2 have very little in common beyond both providing programmatic access to your weather data at weatherlink.com.
v2 offers a more powerful and more comprehensive API than v1 and also, for the first time, provides access to all types of Davis station uploading data to weatherlink.com (data from EM and WLL/WLC stations was not accessible in the past with the v1 API). At the same time, v2 has a more focused approach and not all features from v1 are carried directly across to v2.
For example, no historical/archive data is available in v2 unless the station has a Pro plan attached. In general, the v2 data is more focused on relaying actual sensor readings and related metadata, but without storing a lot of statistical values for different periods – it is up to the user to maintain their own data store elsewhere and to calculate summary values as required. On this aspect Davis say: ‘The WeatherLink v2 API is intended and designed to serve as a raw data retrieval API and not as a data analytics API’.
Weatherlink.com is now a very large data platform with around 100,000 active weather stations around the world uploading data several times an hour (some uploading every minute). There are also multiple types of Davis station, each uploading in distinct data formats. Providing access to this huge volume of disparately-sourced data inevitably results in the v2 API being moderately complex. Security and authentication are also carefully managed. Using the v2 API is not too difficult but does present some coding challenges and is probably not a project for a novice programmer.
The range of different Davis station types able to upload data to weatherlink.com (summarised below) is responsible for the diversity of data types that the v2 API must be able to cater for.
Station types uploading to weatherlink.com
The Davis weather station range continues to expand and now covers eight different types of device able to upload to weatherlink.com: Weatherlink Live; Weatherlink console; WeatherlinkIP; Weatherlink for Windows (WNAS); Vantage Connect; Enviromonitor (cellular);Enviromonitor (IP gateway) and Davis Airlink. Each of these devices has its own data upload characteristics but broadly they fall into 4 categories:
- Davis Environmonitor (EM) stations
- All other types of Davis weather station, ie Vue and VP2 stations uploading in one of two distinct ways:
- via a logger (serial/USB or IP) or Vantage Connect;
- via the 6100 Weatherlink Live device or the 6313 Weatherlink console;
- Davis Airlink
NB While the sensors themselves are common across the Davis range (remembering also that EM stations allow connection of many third-party sensors), it is the upload device to weatherlink.com that largely determines the available data formats that the v2 API can see. Accordingly, it’s important to be aware of whatever upload device your station(s) use when writing download code.
The key difference is that standard VP2/Vue stations uploading via a legacy Davis console/logger are limited to the standard data fields available from any VP2 console-based station, ie as described in the Davis Serial Tech Ref document. In other words, the maximum loggable sensor complement of a VP2 station is fixed and so a single data structure as in the Davis LOOP or archive records can cater for all the sensors that can ever be logged by this type of VP2 station.
In contrast, EM stations are designed as an extensible ecosystem of sensors and there is almost no limit to the number or diversity of sensors whose readings can be uploaded to the wl.com 2.0 platform. EM stations clearly call for a different approach to data-handling and hence for different types of data structures.
The introduction of the Weatherlink Live (WLL) and 6313 Weatherlink console (WLC) devices add a further layer of diversity. WLL can upload data from ANY combination of up eight VP2 transmitters (ie one per channel) so in theory eight ISS units could be uploading through one WLL device. Given that WLL is not limited by the traditional Davis LOOP/archive records structure but allows many more sensors to be monitored, WLL has more in common (in terms of data handling at least) with EM stations. So, although WLL receives data from all the familiar Vue and VP2 sensors and transmitters, it uses a completely different data architecture and upload protocol to weatherlink.com from that used by eg WeatherlinkIP.
Davis Airlink is the relatively new standalone Davis Air Quality sensor and its data is also handled within the overall weatherlink.com platform and is accessible via the v2 API. Where a weatherlink.com station includes an Airlink within its sensor complement, the Airlink data will also be included within any data downloads from the station.
Despite these major differences between EM, WLL and traditional logger-based stations, the 2.0 platform does its best to handle all classes of station via a common set of programmatic access functions in the v2 API. But inevitably there are significant differences in how these work depending on the types of station and data you’re trying to access. For example, legacy devices are very limited in the Health data (wireless reception, battery voltages etc) that they can upload.
The bottom line is that while the v2 API provides a common gateway to the weatherlink.com data, the available data items and exactly how they are accessed, presented and formatted will depend on the types of station and on the upload device. Any program intended to work with multiple station types must also be designed to accommodate these different data structures.
Subscription level
The 2.0 platform recognises multiple subscription tiers, eg Basic/Free and then the paid-for Pro, Pro+ etc. The Basic/Free tier provides access only to current conditions data download options (ie no historical data at all), but since EM and Connect plans are automatically Pro tier, any limitations of the Basic tier apply only to logger-based stations. You may also need to use a higher Pro plan tier to access data more frequently, to have a shorter archive interval or to access other advanced features.
Using the v2 API
Introduction
There is a detailed description and tutorial for the v2 API at weatherlink.github.io/v2-api/ and the extensive detail there is not repeated here. But the notes below do provide an outline introduction to the key points. Anyone wishing to take an initial look at the v2 API but before writing any code might wish to use the Postman utility for testing and understanding API calls to the weatherlink.com v2 API or, for regular users of the Visual Studio Code editor, the Thunder client extension for VSC, which is a neat and simple alternative to Postman.
- All data needs to be pulled from weatherlink.com by means of individual data requests. In other words, there is no way of configuring weatherlink.com to push v2 API data to the user automatically at regular intervals.
- As might be expected from a modern data platform, all responses – whether data or metadata or error messages – from weatherlink.com to v2 API requests are made in JSON format. So virtually of the work in handling the data responses involves decomposing the JSON data into lists/arrays or dictionaries before extracting the individual sensor readings.
- All requests to the v2 API are made as a web GET request using a specially constructed URL. The exact structure will vary according to the nature of the request, eg whether the request is for current or historical data, or for some metadata such as a station list and sometimes (eg when requesting historic data) may need to be updated with each request. As is typical for such web requests, the complex URL effectively consists of two parts – the path to the resource on the server and then multiple key/value arguments to define the exact request and for authentication. Here’s a (non-functional) example asking for the latest current conditions data for a specific station (with station-id = 96230):
- https://api.weatherlink.com/v2/current/96230?api-key=abc2bsjvgvjkhyk4pxzoya3kc0yhm1ik7
- Each web request must pass an authentication test. Your account credentials (v2 API Key and Secret, which can be found in the lower left of the account page of your account at weatherlink.com), are used in making every web request to weatherlink.com and it is these values that automatically determine which stations you can access data from.The api-key parameter must be passed as a query parameter in the request, as in the example above. In addition, an HTTP header parameter containing the account’s API secret must be passed as part of the request. See the v2 API documentation for further details and examples. If the signature is not 100% valid for some reason then your request will not authenticate and you will see only a JSON error message.
- Every station at weatherlink.com is identified with a unique Station ID, which is typically a 5 or 6 digit integer. This Station ID is different from the station’s Device ID (also sometimes referred to as its MAC address) or any other number to found on the console or logger etc. The only way of finding the station ID is to make an initial web request to the /stations endpoint at weatherlink.com and this will return a batch of JSON metadata containing (amongst other account details) a list of stations whose data you can download, along with the station ID for each station.
- The URL in the web request is directed to different resource endpoints on the weatherlink.com servers according to the purpose of the request. So requests for weather data would be made to the /current or /historic endpoints, obviously according to whether you’re asking for current conditions or historical/archive data. The Station ID integer is added as the final part of the path to designate which station’s data is being requested (ie 96230 in the example URL above). But there are a number of other possible endpoints too, including information about stations visible to the requesting account (the /stations endpoint) or the data format to expect from different sensors (/sensor-catalog ).
- Each account at weatherlink.com has access only to data from stations that the account owns or that are shared with it. For private users, this might be only one single station, but larger concerns might own dozens of stations. Your account credentials (v2 API Key and Secret) automatically determine which stations you can access data from – typically only those stations that the account owns.
- There are now a considerable number of different sensor types that can be fitted to a Davis station, the majority accounted for by the many options available for the Enviromonitor stations. The /sensor-catalog endpoint currently lists 255 different named sensors. (Note however that each distinct sensor is individually named and numbered and so there is effectively some duplication across related sensors. For example, each ISS variant with a distinct Davis part number has its own entry in the sensor catalogue.) But each sensor category will return its JSON sensor data in a particular and distinct format and it is obviously important for anyone writing code to access weatherlink.com data to understand exactly which format data will be returned in from a given station. (‘format’ in the sense of exactly which key/value pairs might be expected in the JSON from any given sensor type.) Therefore, one of the first steps in developing a new program is to download a copy of the sensor catalog from the /sensor-catalog endpoint to act as a reference against which station data can be interpreted.
- Within an overall JSON object returned by a URL call, sensor readings are organised by data structures. Many different data structures are defined and the commoner ones identified by number, each corresponding to a particular category of sensor. A full list of these definitions can be found within the sensor catalog discussed above, but there is an overview in the v2 API Github documentation.
- Some data structures such as the ISS-related ones are large and contain many key/value pairs, while others relating to a single sensor, such as a flow sensor, may have only two or three values. Where a station may be fitted with multiple sensors of a similar type then the data object returned will have multiple instances of the same data structure and it is up to the programmer to keep careful track of which is which. To an extent, the sensor readings and the way they are reported can vary with the upload device and so different upload devices are assigned differently-numbered data structures even though they may be reporting data from similar sensors. It’s worth also noting that current conditions and archive calls return data that is markedly different in nature – the current conditions calls tend to have single sensor values but more context in the sense of totals etc over recent periods, while the archive records are largely simpler sensor readings but repeated for multiple time-points (depending obviously on the time-span for which data was requested).
- Decoding the JSON to extract sensor readings clearly has a degree of complexity. But the details are generally well documented eg in the sensor catalog and, provided the coding is done carefully and thoughtfully, shouldn’t present too much of a challenge. Like many such projects, the volume of information may seem a little daunting but in practice and for most stations only a small subset of sensors and data structures will ever be used and it should be easy enough to become familiar with those.
- Stations vary considerably in the sensors with which they are fitted, ranging from a simple Vue station with a minimal sensor-fit through to an advanced Environmonitor station that may be fitted with multiple nodes, each with multiple sensors. So be aware that even a simple current conditions call to a well-fitted station can return a surprisingly large JSON object with many individual sensor values.
First steps
For anyone new to using the v2 API, here’s a suggested outline process for developing a first program:
- Start by looking at the Github tutorial and create some code able to make a call without errors to the /stations endpoint, ie as far as Step 2.5 of the tutorial. This is the big hurdle to overcome and until you can generate the URL successfully then there’s no point in trying anything more ambitious.
- Once you’ve been able to complete the previous step then this will provide the foundation to proceed further. You should be able to see which stations you have access to and each of their Station ID values. Armed with this information you can then proceed to Stage 3 of the tutorial and retrieve a batch of current conditions JSON data.
- At this stage, you might want to retrieve a copy of the sensor catalog by making a call to the /sensor-catalog endpoint to help understand the different data structures returned by the current conditions URL call. Be aware that the sensor catalogue is a pretty large (150k lines) download. You will probably want to save it to a local file and browse through it in in a JSON viewer such as Mitec JSON VIewer or a similar JSON tool..
- Finally you can complete step 4 of the tutorial and retrieve some archive data (assuming you have access to a station on the Pro tier – see the Data Permissions page for full details of station access).
Post your comment on this topic.