Managing Drone Images with uasimg

Package Goals

Like many R packages, uasimg was born to solve a specific practical problem - how to manage thousands of images from drone projects!

Creating maps and 3D models with drones begins with collecting still images taken with a lot of overlap. This produces a lot of individual images. You then feed these images into photogrammetry or structure from motion (SfM) software, which produces a lot more data (that’s another story, uasimg doesn’t help you manage the outputs from photogrammetry).

For better or worse, most photogrammetry / SfM software expects you—the humble user—to have all your images cleaned and organized ahead of time. For a single project, you can move images around manually with Windows Explorer. But this quickly becomes unmanageable when you have lots of projects, and/or a lot of people involved in different parts of the workflow. uasimg has functions to semi-automate data management and make it more consistent.

A second need uasimg addresses is documenting and sharing collections of drone data. This is helpful for managing projects, but more importantly for long-term data archiving and reproducibility. Drone images are the raw data upon which everything else follows. Someone can always repeat the stitching process, but the images themselves can not be replaced or recreated. A system for locating images, and viewing their properties, is therefore needed for sound scientific practice. The next step - sharing the existence of images with colleagues and search engines, is likewise part and parcel of being part of the scientific community.

Lastly, uasimg has several utility functions to help you analyze individual drone images. These include creating world files (so you can view the photos in GIS software) and cropping out the area of overlap (to make a non-photogrammetric photo mosaic).

uasimg does not:

plan drone flights
do any photogrammetry or structure from motion processing
manage the intermediate or final ouputs from photogrammetry software

Universal Step 1. Extract Image Info

Regardless of what you want to do with uasimg, the first step is extracting the location info from a directory of images. This includes the GPS coordinates of each image, as well as image dimensions, camera name, focal length, drone altitude, bearing, and so forth.

You extract metadata from folder(s) of images with uas_info():

library(uasimg)
flt1_info <- uas_info("D:/Pix4D/PtPinole/imgs/wg/wg_flt01", cache = TRUE)

Notes:

As shown above, the object returned by uas_info() should be saved to a variable so you can use it as input in other functions.
If your drone camera isn’t recognized, you’ll get an error message. Don’t panic. Contact the package maintainer to get your camera added, or see the help page for instructions on how to pass the parameters for your camera.
You can pass a vector of folders to uas_info() to extract the image info for all of them at once. Those folders could represent individual flights, or images from a single flight that has been spread across multiple directories.
uas_info() requires a companion command line utility called EXIFtool, which needs to be installed separately. See the README for details.
Setting cache = TRUE is recommended. This will save the image locations so if the command is run on the folder again it won’t have to run EXIFtool again.
Most modern drones record the altitude about the launch point, however older cameras like the GoPro and multispectral cameras like the Sequoia don’t record relative altitude. In these case, you have the option to pass the relative altitude with the alt_agl argument, or simply leaving it out. Omitting the relative altitude is not a big deal. The only thing it prevents is estimating the GSD and image footprints.

Details: Accuracy of the Estimated GSD and Image Footprints

uasimg estimates image footprints and GSD (ground sampling distance, aka pixel size) based on the camera type and recorded height above ground level. Many drones record the altitude above the launch point, so if the study area is flat these estimates are reasonable (although the accuracy of the recorded altitude is not as accurate as the xy location). If the drone does not record the relative altitude (i.e., most multispectral cameras), you can pass the AGL altitude as an argument to uas_info().

In hilly areas, or where the altitude of camera locations was variable, image footprints and GSD will always be over or under-estimated. If the distance from the drone to the ground is larger than the altitude above the launch point (e.g., flying over a valley), the GSD and footprint will be under-estimated (i.e., too small). Conversely when the distance to the ground is smaller than the above launch altitude (e.g., flying over a hill), the estimated GSD and footprint will be over-estimated.

Flight Metadata

A lot of information can be extracted from the image files, including the date, time, GPS location, camera type, focal length, etc. However other metadata fields pertaining to the flight have to be manually provided. These include things like the name of the pilot, a short description of the mission goals, download link, contact info, etc. These are referred to as flight metadata. Entering flight metadata is completely optional, but many of the more useful functions of uasimg, such as creating useful data catalogs and organizing images into structured directory trees, require flight metadata.

There are two ways you can enter flight metadata - as function arguments or text files. The recommended way is to create a little text file in each image folder. This file will be automatically read assuming the file name starts with ‘metadata’ and ends with a ‘txt’ extension (e.g., metadata.txt, or metadata-flt01.txt). The file should be formatted as YAML, which looks like:

name_long: Point Pinole, West Grove, Flight 03
name_short: ptpinole_wg_flt03
description: Flight over west grove of Eucalyptus trees, 390ft, 90% overlap, 75 degrees off-nadir
proj: ptpin
loc: ptpinole
subloc: westgrove
contact: Andy Lyons
pilot: Andy Lyons
data_url: https://drive.google.com/file/d/1ZpMrYY0xxxxxxxxxxxxxxx

As seen in the example above, each line of the text file should start with a field name (without spaces), followed by a colon, followed by the value (this formatting style is also known as YAML). Blank lines and lines that start with # are ignored.

Any metadata field can be added to the file as long as it follows these rules. The built-in Flight Summary page recognizes the following metadata fields:

uas_flds_oem()

##  [1] "name_short"  "name_long"   "description" "proj"        "loc"        
##  [6] "pilot"       "contact"     "uav"         "data_url"    "tags"       
## [11] "notes"

You can manually create metadata.txt files using Notepad or any text editor. Just be sure to save it in the same folder as the images. Alternately uas_metadata_make() can be used to create ‘blank’ metadata.txt files several directories at once:

my_dirs <- c("D:/uas/HREC/Watershed1/Data/2017-01-16_X5/Flight01_1514_1526_400ft",
             "D:/uas/HREC/Watershed1/Data/2017-01-16_X5/Flight02_1532_1540_400ft")

uas_metadata_make(dirs = my_dirs)

Additional notes about uas_metadata_make():

The fields that uas_metadata_make() inserts in the blank metadata text files can passed as , with defaults set by uas_setflds(). Fields can also be pre-populated using . See the help page for details.
Include open = TRUE to open and edit the metadata files in your text editor.
After you edit a metadata.txt* file, you need to (re)run uas_info() in order to use the latest values in other functions. Unlike EXIF data from the image files, supplemental metadata is always read fresh and never cached.

The second way to save supplemental metadata is by passing a named list as the metadata argument when running uas_info(). The fields supported and their uses are the same as before. For example:

metadata_lst <- list(name_short = "ptpinol_wg_flt03",
                     name_long = "Point Pinole, West Grove, Flight 03",
                     description = "Flight over west grove of Eucalyptus trees",
                     contact = "Andy Lyons",
                     notes = "There might be a cloud shadow half way thru the flight")

flt1_info <- uas_info("D:/uas/PtPinole/imgs/wg/wg_flt01", 
                      metadata = metadata_lst,
                      cache = TRUE)

Creating Data Catalogs

Flight Summaries

uasimg can generate simple HTML pages that show the location and properties of images in a folder (sample). These are generically referred to as ‘Flight Summaries’, even though you sometimes have images from multiple flights saved in one folder. Flight summaries may include an interactive map of the camera locations, their estimated footprints, and image thumbnails. Histograms show the distribution of the estimated ground sampling distance (GSD) and forward overlap. These stand-alone HTML pages can be viewed locally, zipped up and emailed to colleagues, or uploaded to a web server. You can also create a ‘Table of Contents’ page for several Flight Summaries using uas_toc() (see below).

Create Flight Summaries with uas_report(). The main argument required is a flight info object from uas_info().

uas_report(flt1_info)

Notes:

uas_report() has several optional arguments you can use to tailor the Flight Summary:

output_dir. Where to save the HTML file(s). The default is a sub-folder in each image folder called ‘map’.
thumbnails. If TRUE, this will generate preview images for the popup windows in the interactive map.
show_gps_coord. Display the image GPS coordinate in the popup windows.
show_local_dir. Report the local directory where the images are saved.
report_title. Customize the report title.
png_map. Create a static png image showing the camera locations (see below).
attachments. Create and link a KML file of the camera locations and/or flight area (minimum convex polygon for all the images) (see below).
open_report. Open the HTML page when done.

If png_map = TRUE, uasimg will attempt to download a background image for the entire flight area from Google Maps or Stamen. This background image will not appear in the Flight Summary, but is useful for other types of previews, including markdown and the Table of Contents (below). To use a satellite image as the background, you must pass a Google Maps API key with the google_api argument or save your API key to an environment variable ahead of time (see the help page for details). Alternately the background image will be a terrain map from Stamen.

If attachments = 'mcp_kml', the minimum convex polygon for all images will be exported to a KML file, and a link to this file added to the Flight Summary. This is a reasonable proxy for the flight area, and can be used to fly the area again, or see where you’ve already flown. Many flight planning apps allow you to import a KML file.

Creating a Table of Contents of Flight Summaries

uas_toc() creates a Table of Contents (TOC) for multiple Flight Summaries created by uas_report() (sample). The main argument in uas_toc() is a character vector of HTML file names (i.e., returned by uas_report()).

The TOC page will include links to the Flight Summaries wherever they may be on your hard drive. The optional gather_dir argument allows you to copy the Flight Summary HTML files and their dependencies (e.g., thumbnail images, KML files) to a single folder. This allows you to create a TOC page for Flight Summaries across your hard drive in a single folder, which can then be FTPed to a web server.

Additional arguments you use to customize the appearance of a TOC page include:

toc_title. Title for the page.
toc_desc. Short description or tag line.
summary_map. Show a summary map of all the flight areas.
header_html and footer_html. HTML files to insert at the top and bottom of the Table of Contents page.
open_toc. Open the TOC in a browser.

Example: Creating an Image Catalog for Several Flights

The following example illustrates how you could loop through a set of image folders, create a Flight Summary for each one, and then compile a TOC for all of them.

## Create a character vector of folders
img_dirs <- c("D:/Pix4D/PtPinole/imgs/eg/eg_flt1_p4p_sw",
              "D:/Pix4D/PtPinole/imgs/eg/eg_flt1_p4p_se",
              "D:/Pix4D/PtPinole/imgs/eg/eg_flt2_m2p",
              "D:/Pix4D/PtPinole/imgs/eg/eg_flt3_m2p",
              "D:/Pix4D/PtPinole/imgs/wg/wg_flt01",
              "D:/Pix4D/PtPinole/imgs/wg/wg_flt02",
              "D:/Pix4D/PtPinole/imgs/wg/wg_flt03")

## Save the metadata from these flights into a single object
flts_info <- uas_info(img_dirs, cache = TRUE)

## Create individual flight summaries
flts_sum <- uas_report(flts_info, show_local_dir = TRUE, thumbnails = TRUE,
                         show_gps_coord = TRUE, overwrite_html = TRUE,
                         open_report = FALSE, attachments = "mcp_kml", png_map = TRUE)

## Create TOC, copying all the individual HTML files to one place
uas_toc(flts_sum, 
        output_dir = "D:/Pix4D/catalog/ptpinole",
        toc_title = "Point Pinole Eucalyptus Survey Flights", 
        gather_dir = ".", 
        overwrite_toc = TRUE, 
        open_toc = TRUE)

Exporting Image Collection Geometry

You can export the geometry from a flight using uas_exp_kml() and uas_exp_shp(). Features include image centroids, their (estimated) footprints, and MCP (minimum convex polygon). Layers are exported as Shapefiles, a common file format for GIS data.

## Export 
uas_exp_shp(flts_sum, ctr = TRUE, fp = TRUE, mcp = TRUE)
uas_exp_kml(flts_sum, ctr = TRUE, fp = TRUE, mcp = TRUE)