Thursday, September 27, 2018

Sentinel-2 imagery from satellite to a browser in less than 24 hours

MapXplorer.com's un|earth:: mapping app now includes a rolling 30-day archive of Sentinel-2 satellite imagery over Australia. This new information layer extends the un|earth:: app's functionality with higher frequency and higher resolution data source, which allows for monitoring landscape changes on a 5 day cycle (2-3 days where satellite flight paths overlap) and for areas as small as local paddocks. New imagery is released daily, within 24 hours of its acquisition.

The un|earth:: app's initial focus was on enabling access to historical Landsat data over Australia that dates back to 1986. With resolution of 25m per pixel and a 16 days revisit cycle, Landsat satellite imagery is an excellent source of information on changes occurring in a local landscape over an extended period of time. However, the latest information is published with a few weeks delay. The new Sentinel-2 imagery layer closes the time gap to just hours post data acquisition so un|earth:: app users have now access to continuously updated satellite imagery as well as historic archives.

Sentinel-2 satellite imagery has resolution of 10m per pixel and is available free of charge from the European Union's Copernicus Program. Geoscience Australia processes the data into a higher, surface reflectance and terrain corrected level product - to eliminate the effects of atmospheric distortions and terrain shading from the underlying data. The so-called "analysis ready" data is then made available for use by third parties in a variety of formats. It all happens automatically and within hours after Sentinel-2A and Sentinel-2B satellites fly over Australia.

MapXplorer.com handles processing and on-the-fly visualisation of that data in an online browser. The result is ready-to-use, colourful images of Australian land and surrounding ocean waters which are created from various band combinations and displayed in full resolution on an interactive map. The images can be spectacular (see more examples on Twitter @AllThingSpatial):

[Bundaberg.Australia - Sentinel 2 image: natural color RGB (bands 4-3-2) - brightness, contrast and exposure adjusted]

MapXplorer.com's advanced, in-browser, real-time processing technology means there is no need for downloading bulky GeoTIFF files and viewing their content on specialised desktop software, one at a time. Our technology simplifies the distribution process of terabytes of Earth observation data, cuts both the cost and time involved in publishing complex spatial information, as well as reduces the level of expertise required from end users to utilise the underlying data. un|earth:: app supports the use of satellite imagery on both mobile and desktop devices.

Current functionality of the un|earth:: app supports only the most basic use cases and is meant to facilitate the introduction of remote sensing principles and theory to the novice users. In particular, image filter presets were created with the objective to enable changing the presentation of displayed satellite imagery between several different effects with just a single mouse click.

So, users can now quickly switch from true colour RGB image, displayed using Light and Bright filter (as the default option), to a more vibrant colour version (by applying Full Vibrance filter) or versions that that highlight specific landscape features such as exposed earth - displayed in bright orange/red colours after applying Inferno filter, or green vegetation - displayed in bright green after applying Pastures & Greens filter.

[Melbourne.Australia - Sentinel-2 image over Open Street Map: inferno image filter (with adjusted contrast from 60 to 40) which highlights town centres/industrial areas (white) and new development areas/ bare earth (red-orange shades)]

Custom controls enable adjustment of the intensity of applied image filtering parameters such as brightness, contrast, exposure, saturation, vibrancy and gamma. This is a particularly useful feature since there is a slight difference between Landsat and Sentinel-2 input data, which means standard preset filters calibrated to Landsat imagery may occasionally result in an excessively bright Sentinel-2 image, so the colours may need to be "toned down" to improve its appearance, as in the following example:

[Bowen.Australia - Sentinel-2 image: Light & Bright filter and with custom filter parameters- brightness:-8, contrast: 20, exposure:-20, gamma: 1.2]

More custom options for mixing and matching bands are coming soon.

MapXplorer.com's un|earth:: app is a proof of concept technology, created and maintained with funds derived from online advertising presented to the visiting public. Please support our sponsors by reviewing and following through on their offers. We welcome any feedback and inquiries regarding sponsoring functionality improvements or custom deployments.

Wednesday, July 4, 2018

Cracking open 30+ years of Australian Landsat satellite imagery archives

Thanks to a recent technological breakthrough, the Australian public is now able to view and interact in real time with the national collection of Landsat satellite imagery. The entire 32 years of archival data - 300TB in total - can now be searched and displayed in the new innovative browser-based desktop and mobile app un|earth::. This app is accessible for free from MapXplorer.com.

[Perth.Australia - Landsat 8 image: natural color RGB (bands 4-3-2) - brightness, contrast and exposure adjusted]
un|earth:: is first and foremost a demonstration of a very innovative technology but its ultimate purpose is to introduce remote sensing analysis concepts to a wide range of potential users. The app is working proof that it is viable to deliver low cost, advanced spatial analytics capability to the mass market. Hence, as of now, just about anyone will be able to benefit from the publicly available satellite imagery archives, and the privilege of utilising this national treasure will not be restricted to only a narrow group of traditional remote sensing data users.

The Australian Landsat archive is a historic record of landscape change over the last 32 years - change which was shaped by events like droughts, floods, bushfires, cyclones, land erosion but also urban expansion, land clearing, mining, crop growing, grazing and similar human activities.

The collection contains images of Australia acquired by Landsat 5, Landsat 7 and Landsat 8 satellites since 1986. There are over 3 million individual pieces of data to browse through. The archive is still growing, with imagery acquisition occurring up to six times a day over various parts of Australia every single day. Updates are released to the public on a regular basis, so the latest imagery is no more than just a few weeks old.

With pixel size of 25m, Landsat satellite imagery is ideal for local, regional and continent-scale analysis. A unique feature of the Australian collection is that the data is distributed in an "analysis ready" state - which makes it easy to compare the data through time and between various locations, but also with data from other satellites and with various resolutions. This is unlike other public Landsat data versions available directly from the USGS or through other third parties.

[Cowra.Australia - Landsat 8 image over Open Street Map: fractional cover - red: bare soil, green: photosynthetic vegetation, blue: non-photosynthetic vegetation]
The Australian Landsat archive is managed by Geoscience Australia and is accessible via the National Computational Infrastructure public website under the Creative Commons Attribution licence. Which means anyone can make a use of this data and for any purpose - whether it is public, commercial or private.

The data has been available for download for some time now but not in the synthesized format which is ready for immediate online use. A break-through innovation implemented in the un|earth:: application makes it possible to access and analyse all that historical Landsat data with just a web browser.

Experience un|earth:: first hand by visiting MapXplorer.com - more information about its current functionality in a recent media release.

Tuesday, June 27, 2017

2016 Census – here comes the data!

Tuesday 27 of June 2017 is the official release date of 2016 Census data. The first batch of information contains a comprehensive list of demographic and socio-economic topics for areas down to local neighbourhood level.

The ABS Census of Population and Housing is conducted every five years and provides a comprehensive picture of Australia's people, how they live and where they reside. The information collected during the Census is used to distribute government funds and plan services for the community – including those relating to housing, transport, education, industry, hospitals and the environment.

Census data is also used by individuals and organisations in the public and private sectors to make informed decisions on policy and planning issues that impact the lives of all Australians. And of course, Census data is also a valuable source of statistics for research and data analysis in support of business activities as it provides comprehensive information on potential customers and markets.

MapDeck will be adding the 2016 Census data to its inventory progressively from the day of its official release, starting with free excerpts of statistics for Postal Areas.

All relevant Census 2016 related statistical geography is already available on MapDeck for immediate perusal. This includes, for example, Mesh Blocks, the smallest statistical geography units and building blocks for all other larger structures, like suburbs or postal and local government areas, etc.

A unique feature of Mesh Blocks is that each one is classified according to a dominant use of the area, like residential, parkland, education, commercial, etc. This makes it possible to use mesh blocks to create simple maps of land use, as the one below for the ACT.




Imagine the possibilities… Sign up for free MapDeck account today (your invite code is: c3df6244 )!


Map: ACT Mesh Blocks by Category, 2017

 

Tuesday, April 11, 2017

Small area population estimates for Australia

The Australian Bureau of Statistics has recently updated its population estimates for Statistical Areas Level 2. This data set tracks 25 years of population changes in Australia back to 1991. The data is current as at 30 June 2016.

The Estimated Resident Population data set can be considered a prelude to the upcoming 2016 Census data release in late June 2017. It is still based on the 2011 edition of the Australian Statistical Geography Standard (ASGS2011) so, the figures for the SA2 areas will differ from the 2016 Census data, but ERP data is a good proxy of what to expect.

According to the ABS estimates, in 2016 there were 24,128,876 individuals residing in Australia, with the majority of people settled along the coast line. An interesting pattern of change over time is that, while the population in coastal areas continues to grow, there is long term depopulation occurring in the red centre - as depicted by blue polygons on the map below.

Population change 1991-2016 (blue - decline; orange and red - increase)


However, there is a notable positive change in the population of the Northern Australia, with some areas close to doubling its population in the last 25 years.

While population density in coastal areas has increased significantly over time, population growth is not uniformly distributed. Zooming in on capital cities allows pinpointing which areas have grown faster than the others.

For example, in the case of Sydney, the areas that stand out include Sydney City – Pyrmont – Ultimo, with a growth factor of around 9 (where 1 = no change and values below 1 indicate decline in population).

But the highest growth factor of 37.5 was recorded in Glenwood in the North West, followed by Parklea - Kellyville Ridge with 21.0 and Hoxton Park - Horningsea Park with growth factor of 12.7 over the last 25 years.

Population change by Statistical Area Level 2 (SA2)

An interactive map featuring several more data layers can be explored on MapDeck.com (please use this code to create a free account: c3df6244).

MapDeck.com Products List:
Population Estimates by SA2, Australia 2016 (free data table)
Population Estimates by LGA, Australia 2016 (free data table)
Estimated Resident Population 1991-2016 (free interactive map)




Thursday, March 30, 2017

2016 Census Data is Coming!

On Tuesday 11 April 2017 the Australian Bureau of Statistics (ABS) will release a teaser, or snapshot if you like, of what attributes make up a ‘typical’ Australian in 2016. This information will only be available at the national and state/territory level. The first detailed 2016 Australian Census data will be officially released on Tuesday, 27 June 2017.

In preparation for this occasion, MapDeck.com is making its 2011 version of the Basic Community Profiles for Postal Areas dataset freely available. This free data release gives MapDeck.com users a chance to experience its unique capabilities of custom-map creation and personalised in-map analytics – and get ready for the avalanche of information that will be published in coming months!

It is a “big data set about small areas” - in total there are 7,942 individual data items, spread across 66 tables, which can all be used for mapping. There is also a companion data set in Excel format which helps explain the structure of the data and what is specifically available in individual tables.

Postal Area Boundaries 2011 and 2016 are already available for free perusal on the MapDeck.com platform and both versions can be conveniently compared on the Australian Postcodes map.

Below you will find links to several public maps - they are published as examples of what can be created with MapDeck’s Thematic Mapper app:

Population by Postal Areas, Australia 2011
Dwellings by Postal Areas, Australia 2011
Selected Averages by Postal Areas, Australia 2011


Proportions of females to males by Postal Area, 2011


And you can experience the full in-map analytical capabilities and create private maps by subscribing to the Thematic Mapper app for 30 days for just a U$100 (thereafter, any maps you create can be viewed with free Thematic Mapper Basic app). 

Learn how to create a map by following these few simple steps (login required): How to create maps with coloured polygons in Thematic Mapper


Get the tools ready! Brash up on map making skills and start analysing the data for better decisions!

Your invitation code to create a free MapDeck.com account is: c3df6244

Tuesday, March 7, 2017

Dissecting Australian Property Investors

Is Sydney the epicentre of property speculation in Australia? ATO data provides a surprising answer…

A lot has been said about Sydney property investors in recent years, mostly in derogatory terms. Commentators usually criticise their financial irresponsibility, or even sheer madness, for “piling into the residential property market with excessive leverage” - something which is said to inevitably end badly for each and every one of them. But are these opinions justified?

The Australian Taxation Office (ATO) data summary of 2013-14 individual tax returns for postcodes, the latest available, paints a surprisingly different picture.

In particular, it is true that property investment is quite popular amongst Sydneysiders. This is clearly illustrated on the following map - purple shaded polygons mark postcodes with the above average proportions of property investors amongst the residents, comparing to figures for Australia as the whole.

Postcodes with the above average proportions of property investors, 2013-14
Source: Australian Investor Profiles by MapDeck.com

However, the big surprise revealed in the ATO data is that a significant proportion of Sydney-based property investors are actually netting hard cash from renting out residential properties. That is, their residential property portfolios are not negatively geared at all!

The following map points out postcodes in the Greater Sydney area with a positive average net rental income per investor. There are quite a few of them. So, it turns out, Sydney-based property investors are not leveraged to the hilt en masse as is commonly assumed.

Postcodes with a positive average net rental income per investor, 2013-14
Source: Australian Investor Profiles by MapDeck.com

There are actually only a handful of postcodes in Greater Sydney with residents who very aggressively gear their investment property portfolios. That is, postcodes which have above average proportions of property investors, and where residents report rental losses at above the national average level. These are marked on the following map as areas with red background.

Property investors by Postcode 2013-14
Source: Australian Investor Profiles by MapDeck.com

And of those, there are only six postcodes where 6,268 property investors (out of the total of 2,672,004 taxpayers counted in the Greater Sydney area) potentially fit the common stereotype of a property speculator. That is, property investors residing in these postcodes invest in rental properties in above average proportions comparing to the national average and have no or very limited other investments (i.e. are categorised as “Property Centric” investors). AND, at the same time, they are also highly leveraged (i.e. report above average rental losses to the ATO).

These are postcodes 2745, 2768, 2769, 2171, 2173 and 2231 - marked on the map as red polygons with black horizontal stripes.

Concluding, it is very unlikely that things have changed dramatically in the last two and a half years. So claims of irresponsible, mass scale residential property speculation by Sydneysiders are rather exaggerated. In fact, it appears that quite the opposite is true.

Only aggregated data is available for analysis, nevertheless, it suggests that large proportions of Sydney-based taxpayers are playing very safely with residential property investments and are generally diversified into several investment classes, including shares, cash and equivalent.

A full set of ATO statistics for postcodes, with extended investor profile information, has been published by MapDeck.com and is accessible online.

===========

About MapDeck

MapDeck empowers individuals and organisations to make better choices and more informed decisions. By providing easy access to a wealth of statistical information and simple to use location intelligence tools we help our clients in reaching their goals sooner, succeeding faster and on a grander scale.

Invitation code: c3df6244 for a free service.

Friday, December 16, 2016

Australian generosity mapped – how charitable are we really?

Many Australians are happy to give a helping hand to various worthy causes throughout the year. Indeed, each year Australians hand over an impressive $2+ billion in tax deductible gifts and donations, and probably a bit extra in non-declared funds. But collectively, are we as generous as we claim to be? The facts may surprise you.

A new analysis of 2013-14 taxation data by MapDeck.com sheds some very interesting light on the donations habits of Australians. It provides a detailed picture of charitable donations made across neighbourhoods of our eight capital cities: Adelaide, Brisbane, Canberra, Darwin, Hobart, Melbourne, Perth and Sydney. MapDeck also provides a State/Territory specific view of donations for cross-border comparisons.

Looking at the findings first at a national level, it is encouraging that almost one in three Australian taxpayers is actively donating money for charitable purposes. However, a significant proportion of donations – amounting to 37% of the value of all donations – come from only 5.3% of taxpayers. These taxpayers live in 195 postcodes around Australia categorised as the ‘most generous’ in terms of the rate of donations per local population, as well as the average value of donations. In the 2013/14 tax year, these 677,093 individual donors collectively contributed a billion dollars for charitable purposes (or, on average, $1,416 per donor).


The remaining 3.8 million donors provided $1.6 billion of the total $2.6 billion donated that year (or, on average, $424 per donor). In addition, the fact remains that almost 8.3 million taxpayers (or 65% of all Australian taxpayers) reported no donations at all. So, it appears that Australians are not so charitable after all…

The level of generosity of donors differs significantly from one location to another. So, where do the most generous Australians live?


New South Wales was the state with the highest number of donors (1.45 million) but Victoria was not far behind, with just over 1.2 million. Interestingly however, the generosity of NSW donors far exceed that of Victorians – NSW residents donated $1 billion that year, a whopping 45% more than Victorians.

The average value of donations per donor was $707 in NSW and only $571 in Victoria. However, 38% of Victorian taxpayers made tax deductible donations, comparing to only 36% of NSW taxpayers. So, while NSW donors appear more generous, donating for charitable purposes is slightly more prevalent among Victorian taxpayers.

With exception of the ACT, the rate of donation and average values per donor for the other States and Territories are substantially below the two most populous Australian states.

Given the distribution of the Australian population, it is not surprising that almost 82% of all donations come from capital-city residents. The average donation per capital city resident was $655 in 2013/14, comparing to an average of $573 for all Australian donors.

On a city by city level, close to 40% of all gifts and donations reported to ATO in the 2013/14 financial year came from Sydney residents ($848 million), while residents of Melbourne contributed only 27%, or just over $607 million. Sydney, Melbourne and Brisbane accounted for a combined total of 80% of reported gifts and donations, while the remaining five capital cities made up only 20% of the overall sum.


Sydney-based donors contributed on average $872 in gifts and donations in 2013/14, and were followed by Melbourne and Canberra donors with average donations per donor of $638 and $603 respectively.


But which of our Australian capital cities has the most engaged community in terms of contributions to charitable causes? It turns out it is Canberra - the nation’s capital, filled with hardworking public servants and households with the highest average incomes. The city had the most generous population with 45% all tax paying individuals reporting gifts or donations on their statements. This compares to 39% for Melbourne taxpayers and just 36% in the case of Sydney taxpayers.

Hobart stands out amongst the smaller cities with 38% of taxpayers making tax deductible gifts and donations. Brisbane with Gold Coast and Perth closed the list at 33% each.

Putting the data published by the ATO through a slightly more advanced analysis can reveal even deeper insights about the generosity of Australians. For example, by comparing gifts and donations claimed by individual postcodes with averages for capital cities or Australia as a whole, it is possible to create a simple classification that reflects common patterns of behaviour across local communities.

Looking more specifically at the four categories of donors, of the most generous donors (that is, those that donate above average amounts and in above average proportions comparing to the national average) 42% live in NSW. Out of the 677,000 most generous Australian taxpayers, this amounts to a total of 287,000 individuals. Only 30% of the most generous donors, or 204,000 individuals, are located in Victoria.


However, 40% of the second most generous group of donors (that is, those who contribute at higher than average rates but tend to donate below average amounts) are from Victoria. NSW has only 34% of all donors in this category.

Without trying to fuel the animosity between the two states any further, this seems to be a typical pattern that distinguishes NSW residents from Victorians across several measures analysed in this study. That is, when NSW residents, and Sydneysiders in particular, decide to give, they give generously. Much more generously than residents of any other capital city or State or Territory in Australia. However, donating to charitable causes is more widespread amongst the population of Victoria.

Nevertheless, it is interesting to note that the postcode with the highest donation per donor was actually in Greater Melbourne area: it is 3944 (Portsea) with an average donation per donor of a whopping $19,187. Sydney postcode 2027 (covering the exclusive suburbs of Darling Point and Point Piper) with an average donation of $14,180 per donor is only third on the list - after Brisbane postcode of 4009 (Eagle Farm) with average donation of $18,490.


The following maps show how average donations compared across Greater Melbourne and Greater Sydney postcodes.


Map 1: Average Donations by Postcode – Greater Sydney, FY13/14


Map 2: Average Donations by Postcode – Greater Melbourne, FY13/14

The richness of information contained in this ATO data also makes it possible to investigate some more controversial aspects of our generosity. For example, pinpointing postcodes where residents earn the most but donate very little. There could be many colourful ways of describing the residents of such postcodes but, for the purposes of this article, let’s use a less judgemental label for them, such as “Well-to-do non-participants”. Which State or Territory do you think has the most residents in this category?

Log into your MapDeck.com account and find out!  A full deck of interactive maps is accessible to all registered MapDeck users via the links listed below. You will need this invitation code to create a free account if you are not yet a MapDeck user: f10dc1f2 .

And if you wish to explore the entire Generosity Profile of Taxpayers by Postcode, Australia 2014 data set and draw your own conclusions, or if you would just like to identify where those 195 postcodes with the most generous donors are located, it is available for a small fee for download in Microsoft Excel format and/or as a subscription for use with MapDeck’s Thematic Mapper app.

Implying that “Australians are charitable folks” would be an overstatement - the evidence summarised in this article points rather to the contrary. True, Australians are one nation but we tend to differ quite significantly as individuals, including how generous we are with gifts and charitable donations. However, not to affront our national pride, let’s draw the final conclusion borrowing from George Orwell’s famous quote. That is: it is quite likely all Australians are charitable by nature but some of us are more charitable in practice then others.


About Generosity Profile of Australian Taxpayers

The profiles area based on the Australian Taxation Office statistics for residential postcodes sourced from 2014 individual income tax returns processed by 31 October 2015. The statistics are not necessarily complete.

The information can be used for marketing and promotional purposes, or for profiling existing or prospective customers based on their place of residence.

Map (free):  Gifts and Donations by Individuals, Australia 2013-14
Map (free):  Gifts and Donations: Well-to-do Non-participants, Australia 2014
Data (free): ATO Gifts and Donations by Individuals, Australia 2014
Data (paid download): Generosity Profile Table, Postcodes, Australia 2014
Data (paid subscription): Generosity Profile of Taxpayers by Postcode, Australia 2014


About MapDeck:

MapDeck’s mission is to empower individuals and organisations to make better choices, more informed decisions – so they can reach their goals sooner, succeed faster and on a grander scale.

MapDeck is an online marketplace for spatial information and simple-to-use, task-oriented tools to support a variety of activities, be it business or investment related, environmental, community or policy focused.