Image analysis software

In the field of remote sensing image analysis, there are 3 main components which have to be combined in order to achieve good results. These are: data, software and thematic expertise.

Even though one may think that software is just a tool, this particular tool is paramount when it comes to deal with the huge amount of information contained in the images. Fortunately there exist number of commercial tools available to choose from. Just try searching on Google with "remote sensing image software" and you will see.

These commercial packages offer a bunch of interesting functionnalities allowing to perform very interesting processing tasks on images. They also offer the support from their editors. Therefore they are a good choice for many users.

There also exist free software packages: free in the sense of gratis but also free in the sens of allowing you to access the source code and modify it to suit your needs. I will not try here to advocate for the use of free software, even if my preference goes there.

However, when new types of sensors become available, the end user suffers from a long delay between data availability and software support for the new type of data. I am not talking about image formats or meta-data access. Usually software editors are proactive and provide these kind of functionnalities rather quickly. I am talking about algorithms and techniques developed in the research labs and which allow to exploit the specificities of the new data.

The usual way of proceeding is the following: a lab develops an algorithm and writes some code in order to validate it on test data, then comes the publication under the form of a journal article. After that, software editors have the tough work -- and responsibility -- of deciding what can be interesting for users, code it and validate the new algorithms on big amounts of data before making this available to their users. This procedure is long and expensive and constitutes an important risk for software editors.

In many research fields -- other than geoscience and remote sensing -- scientists are eager to share data and software in order to intervalidate and compare different approaches (see the Public Library of Science or the Insight Journal, for instance). That means, that, together with the publication, the source code and the data used for the publication is made available under licences which allow the use, the modification and the distribution of those.

This approach allows a rapid evolution from concepts and prototypes to operational systems available to end users. This is not a bad thing for software editors, since they have found the way to integrate this kind of free software in their commercial packages (via plugins, external modules, etc.). This also opens up a market linked to education, application tuning for specific users, etc.

Of course, scientist in labs are not professional software developers, so the code they make available needs to be improved and optimized. Free software allows people to look how the code is made, so everybody can find bugs and correct them. Furthermore, in many fields, software editors themselves contribute to free software develoment, because they need this approach in order to provide interesting functionnalities for their users. Some of them are even sponsoring initiatives as the OSGeo Foundation.

In the field of remote sensing, most of free software projects are related to the data access and the pre-processing steps (see here for a list of projects) and there are few projects whose goal is providing tools for information extraction. This is no doubt due to the fact that these tasks may be specific to each application and different to automate. At the other end of the information production chain, there are lots of free software tools for mapping, GIS, etc.

At CNES, in the frame of a program called ORFEO which aims at helping future users of the Pléiades and Cosmo-Skymed systems to get ready to use the data, we are developing a set of algorithmic components which allow to capitalise the methodological know how, and therefore use an incremental approach to take profit of the results of the methodological research.

The ORFEO Toolbox (OTB) is free software and can be downloaded here.

OTB is based on the medical image processing library ITK and offers particular functionnalities for remote sensing image processing in general and for high spatial resolution images in particular.

Currently, OTB provides:

- image access: optimized read/write access for most of remote sensing
image formats, meta-data access, simple visualization;
- filtering: blurring, denoising, enhancement;
- feature extraction: interest points, alignments, lines;
- image segmentation: region growing, watershed, level sets;
- classification: K-means, SVM, Markov random fields, Kohonen's map;

- multiscale analysis;

- spatial reasoning;

- change detection.

Many of these functionnalities are provided by ITK and have been tested
and documented for the use with remote sensing data. We also use other libraries as GDAL, or OSSIM.

We have some tenths of users and we expect that the toolbox will receive contributions from reserachers and end users.

Multi-resolution change detection

Yesterday I was at the PhD thesis defence of Amandine ROBIN. The subject of the thesis is "Sub-pixel classification and change detection using multi-resolution remote sensing images" and the work has been co-funded by CNES (the French Space Agency) and EADS-Astrium.

The goal of the work was to develop techniques able to combine the high temporal repetitivity (1 day to 10 days) of low resolution (100 m. to 1 km.) sensors together with the high spatial resolution (10 m. to 30 m.) of sensors having longer revisit periods (from 30 days to several months). This means trying to combine SPOT XS images with MeRIS data, or Landsat with SPOT VGT.

In terms of application field this is very interesting, since, for instance, in the case of precision farming, one may want to update a landcover map several times in one season, but due to cloud cover issues it may be difficult to obtain a series of high spatial resolution data with a good temporal sampling. In this case, the approach proposed in the thesis was to use a high resolution geometric reference for the regions of the study area (the agricultural plots) and perform a classification of each region using the low spatial resolution temporal series.

Typically, this means using a SPOT XS image acquired at the begining of the season in order to perform a segmentation (the classification is nearly impossible with one single date, since most of the classes are distinguished by their temporal evolution rather than by their radiometry at a given date). Then, using several MeRIS images acquired during the season, the classification may be performed. It is interesting to note that the final classification has the SPOT resolution (20 m.), but the temporal series has pixels which have a size of 300m. by 300m. (resolution factor of 15 by 15). Amandine has shown that the results are good up to resolution factors of about 40 by 40.

The other application was detecting changes between a low spatial resolution temporal series and a high resolution reference classification. That means that you have a high resolution classification obtained with the method presented above and using low resolution images (even a single acquisition, no need for a long temporal series) you can detect which low resolution pixels have changed. This change alert can be used to trigger a new classification or even to schedule a high spatial resolution acquisition.

Amandine's work is based on a sound theoretical approach and the results can be applied to optical sensors either having the same spectral bands between the high and low resolution data or even having different bands but allowing to compute the same vegetation index (NDVI, Leaf Area Index, etc.).

I am persuaded that these results are ready to be used in operational chains.

Amandine's homepage has some material available for download which is worth looking at.

Foreword

Since the publication of the Google Earth software, many people have discovered what remote sensing is. Of course, Google Earth has also open the possibility of building new services and applications.

Even if I think that Google Earth may be very useful (like a map, a world map, enriched, on top of which everybody can publish information), there is one thing missing: up to date images and multi-temporal data.

Many applications of remote sensing imagery are oriented towards monitoring: landscape evolutions, change detection, etc. For this kind of applications, several images of the study site are needed. Furthermore, these images must yield the kind of information the user is looking for. This will determine the type of sensor to use: spectral bands, resolution, and so on.

Once the images are available, they have to be precisely calibrated: artifacts due to point of view, atmosphere or even changes at the sensor level have to be accounted for. Since each image may contain several millions of pixels, this calibration can not be performed manually. Therefore, computer programs have to be written in order to process the images automatically.

Also, the image processing needed for each application may be different and thus the algorithms may need to be tuned, that is, some parameters may have to be accurately chosen.

If the choice of algorithms and parameters is rather well known for some applications, there is still a need of research and development for many other uses. In most of the cases there is no need for a theoretical research, but rather an engineering work of getting the pieces together in order to build operational image processing chains.

My goal in writing this blog is to discuss about examples of applications and services which can be built upon remote sensing imagery and automated thanks to image processing techniques.