ESCI497Z:
UAS for Environmental Research and Monitoring
Post
Point Heron Colony Survey; Introduction to Agisoft Metashape
Updated
10/10/2022
Objective:
We
will be using imagery acquired using our 3DR Solo to survey the Great Blue
Heron colony at near Post Point. As you may know, each year about 10-20 Great
Blue Herons nest near Post Point. The colony is not active during the fall
(October) but we’ll see if UAS provide a viable method for locating nests. This
exercise will familiarize you with the use of Agisoft Metashape
(AM) software. Note that the older version of this software was called Agisoft Photoscan so at times in this document I may screw up and
refer to Photoscan when I’m really talking about Metashape. And, at times we will be using documentation
from the old Photoscan software since they have not
released an updated tutorial for Metashape. Metashape has some improvements and the interface is a bit
different than Photoscan. Anyway, the AM software
will enable us to create an orthomosaic from all of our images and it will also
enable us to create a Digital Surface Model from this imagery. AM is comparable
to Pix4D, Drone Deploy and Drone2Map, an ESRI product that just came out but
I’ve not yet had a chance to use. All of these are “Structure from Motion”
(SFM) software packages and not sure, but it think they may all be using
algorithms purchased/licensed from Agisoft.
PROCEDURES:
Step
1: Get the data. Go to J:/Saldata/ESCI-497/post_pt_heron.
Copy the entire post_pt_heron folder and then go to
C:/temp on our computer, create a folder with your name and paste the post_pt_heron folder there. So, you should now have a
folder on your computer called C:/temp/yourname/post_pt_heron
that contains everything from the post_pt_heron
folder on the J-drive.
Step
2: Open Agisoft. Graze through the vast maze of software on
your computer and find Agisoft Metashape and open it.
This will bring up a screen that looks like this.
In the lower left corner of this window, click on the References tab to bring up this:
MAlso,
go the Post_pt_heron folder and open the PS_1.3-Tutorial (BL)-Ortho, DEM (with
GCPs).pdf file. We will be following this tutorial (mostly) but I’ll add a few
comments/clarifications as we work through it. As noted above, this is a
tutorial for the older Agisoft Photoscan software and interface for Metashape
is a bit different. I’ll try to walk you through the differences.
Set
Phtoscan Preferences: You can skip the stuff in this
section. Defaults should be fine.
Step
2 Add Photos
Our Post_pt_heron folder includes a subfolder called
/120mflt. This includes about 75 images acquired on a programed flight over the
study area at an elevation of 120m (~400 feet). Here is an overview of the area
covered by the imagery:
OK, now follow the tutorial instructions to Add Photos. When you do so, you will
note that all of the images appear in the Photos
section (lower right) of the AM screen and a list of all of the image names
appear in the Cameras section (upper
left). Unfortunately, the GPS in my camera wasn’t working when I did this
flight so there are no X, Y Z positions for the cameras. If the images had
included geotagged, this information would appear in the Cameras section along with the image names. In some cases, you
might have geotags for the images in a separate file and you could load these
following the instructions. If you do have geotags for your images, a bunch of
blue dots would appear in the grey Model
window showing the relative position of each image.
But we can move forward without this information……so just skip all of the Load
Camera Positions instructions.
Step
3: Camera calibration
All cameras and lens have distortion to some degree.
You can use an auxiliary program, appropriately called “LENS” to gather a bunch of obscure parameters to characterize your
particular camera. I will not pretend that I understand what much of this but
getting this info for your camera improves the quality of your results. It is
pretty straightforward. Using the LENS software,
you display a checkerboard pattern on your computer screen, take a few photos
with your camera, load the photos into the software, push a few buttons and,
SHAZAM, you get a camera calibration file.
You can proceed WITHOUT a camera calibration file.
Without one, the software will derive one using the imagery but you get a
better result if you have one.
So, follow the tutorial instructions to
load the camera calibration file (wwucolor2.xml). Click on the folder icon to
load this file.
Note that the interface in Metashape is a
bit different than the Photoscan interface shown in the tutorial. After loading
the hit OK to close the Camera
Calibration dialog box.
Step
4: Align Photos
This is the first really time consuming
step. In the Align Photos dialog,
set the Accuracy to Medium and make sure that Generic preselection is checked. You
would use Reference preselection if your images were geotagged and this would
help speed up the alignment process since the software would have a rough idea
of which photos were near each other. With Generic preselection, the alignment
process takes longer since the software doesn’t have this information. Click on
the Advanced pull down to see more
options.
Key
points
are locations in an image where there is some variability. These represent
unique features in each image which could be useful in matching up photos. You
want as many as possible in each image but setting this too high can result in
a large number of unreliable points. Go with 60,000.
Tie
points are
key points that match across multiple images. Using a value of 0 does not set
an upper limit (which seems rather backwards I know) so go with a value of 0
Click on OK to begin the alignment process. This should take about 5 minutes
to run.
Be patient. Recognize that the software is
looking for “tie points” – obvious features – on every image and looking for
matches on every OTHER image in the dataset. This involves bazillions of
calculations and comparisons to align all of the images.
When it finishes running, you will see
something like this in the Model
window. This is what is known as a “sparse point cloud.”
You may need to resize all the sub-windows
to see this. And, if you click in the Model
window, you can use your mouse roller to resize the image.
If you do not see this, don’t
panic. It’s time to learn a few tricks
Toggle
camera locations on/off:To make the blue squares appear (or disappear), click
on the camera icon on the toolbar at
the top of your screen. Each thumbnail represents the approximate footprint of
an image.
Image
rotation: Also
note the rather ghostly sphere in the center of the image and the little X, Y,
Z axis in the lower right. You can rotate the image in 3 dimensions by placing your
cursor in the screen somewhere, push and hold the left mouse button,
and drag your mouse around. Note that your image might be UPSIDE DOWN when it
first appears. By that I mean that you may be viewing your image from
underground. Weird, I know
Try to get a somewhat side view of your
image. With the thumbnails on you should see something like this:
The small black line indicates the
position the camera was pointing (usually straight down but sometimes at a
slight angle).
Zooming:
Use
the roller button on your mouse to zoom in or out. Note that as you zoom in,
you will see the individual names appear. These disappear as you zoom out.
Moving
the image: When
zoomed in, you may want to examine a different part of the image. To move
around, push and hold the right mouse button, and drag your mouse
around.
Sparse
Point Cloud:
OK with the thumbnails turned off, what ARE those green and white and blue dots
in your image? While aligning the images, AM created what is known as a sparse
point cloud. This consists of a modest number of points (in this case
only 109,337 points) each with an X, Y, Z coordinate and a color derived from
the photos. A bit later, we’ll create a dense point cloud with millions of
points that will provide us with a much better 3-dimensional representation of
our study area.
Before we can do this, we need to locate
some ground control points (GCPs) in our images. AM refers to GCPs as Markers.
Step
5: Place Markers
SAVE
YOUR WORK SO FAR!
To save your work thus far, go to File-Save As and give your project a
file name. This will create a file with a .psx suffix. If you wanted to quit at
this point, you could come back and open this .psx file and you would come
right back to where you left off. You should get in the habit of saving your
work periodically as you move through this project. Note added 4/24/2020: If you take a look at the file size for your
*.psx file, you will note that it is quite small, perhaps 1 Kb or so. This file
is nothing more than a bunch of pointers to a bunch of larger files. All of
these larger files are contained in a folder with the same name as your *.psx file. This means that at the
end of a work session, when you move your files from your workspace in C:/temp/yourfoldername you should be sure to move note just the *.psx
file but also the folder with the same name.
Step
6: Place Markers
Ideally, I would go out to the site before
flying with a VERY good GPS unit and, log X, Y, Z coordinates for 10-12
locations and mark these locations in a way that would be very visible in the
photos.
A quick and dirty approach: If you do not need
to line up imagery with high precision, one quick and dirty approach is to just
use Google Earth. Zoom in on the imagery, find a prominent feature and record
the coordinates from Google Earth (latitude, longitude and elevation). This
will probably get you to within a meter or two and this will be good enough for
our current effort.
So, start this process by following the
tutorial instructions to Build Mesh. But
in the Build Mesh dialog, be sure to
set the Source data to Depth maps
and be sure to set the Quality and Face count to Low to reduce
processing time.
Click OK
to run this. This just takes a few minutes and will create something like
this:
You can get back to your sparse point
cloud image by clicking on the icon on the top toolbar.
I have coordinates for five GCPs that we
can use for our study area. More would be better but this will be sufficient
for our purposes here. The first two are on the south side of our study area on
residential streets. (Note that I DID NOT fly over this residential area but the field of view of the camera was
such that I did get a bit of coverage of a handful of houses). Here are the
rough positions of the first two GCPs (this is an image from Bing, not my UAS
imagery).
Starting with GCP#1, go to AM and zoom in
a bit on this part of your image. Make sure that you have the camera locations
turned on. Zoom in enough so you can see the image labels. Make a guess
regarding which image might include this GCP.
I’m guessing that this GCP might be
visible down near the turn at the SE corner of the study area.
So, this looks like it might be image
#8734. Down in the Photo section of
the screen, scroll down to find this image. When you do, double-click on it.
This will bring up a second tab in the upper right section of the screen that
will display this image. Like this:
Note that you will now see two tabs in the
upper right screen, one for the Model
(the thing that you were viewing before) and one showing image #8734. And,
GCP#1 is right in the middle of this cul-de-sac.
So, follow the tutorial instructions to
place your first “Marker” right dead
center in this cul-de-sak. You can use the roller button on your mouse to zoom
in on the image. And you can left-click and hold to move around in the image.
So, here is my first marker:
Yes! I know that the marker placement
feels a bit arbitrary in this case but you can probably get in placed within
~0.5 m or so of the center. If you need to adjust the placement a bit, put the
cursor on the marker, left-click and hold and drag it a bit.
OK, so you have located Marker 1 on a
single image. Now we need to locate it on every other image where it is
visible. Start by looking at the next image. In the Photo section of the screen, double-click on image #8735. You will
note that there is a blue flag in the cul-de-sac. AM has made a guess as to where
Mark 1 is probably located in this image. You should zoom in and check it. You
may decide to drag it just a bit….or not. But you should click on the marker
even if you choose not to move it. When you do, the flag will change to green
indicating your confirmation of this position.
Now you need to check ALL the other images
where this marker if visible. AM can narrow your search. On the middle left of
the screen, you will see a panel that says Markers
and point 1. Right-click on point 1 and you will see an option to filter photos
by markers. Select this and you will now see that there are only seven image in
the Photos panel. Click through each
of these images and confirm the marker location in each one.
After doing so, click on the icon in the Photos panel to go back to viewing all
of the images.
Find
GCP#2
Click on the Model tab in the upper right panel so you can see the entire study
area. Make a guess on the location of GCP#2. Try image #8747. This is a tough
one since there are quite a few trees. Here is the location for this one:
As for Marker 1, find this spot in the
next image, then filter photos by marker. This yields about 17 images that
potentially include this marker. However,
the trees will block your view of the road surface in some cases, like this
one:
In this case, you should right-click on
the point and select Remove marker. Doing
so will remove the blue flag and replace it with a funny looking grey thingy.
Continue checking all of the other photos
for Marker 2.
GCP#3-5
Here are the general locations for the
remaining GCPs
OK, now add markers for each of these 3 points. Points
3 & 5 are dead center in these trail intersections. Point 4 is the
northeast corner of that white square. This is an example of a really really
good choice for a GCP in that the exact location of the GCP is very obvious in
the imagery. Admittedly, correctly locating the other 4 GCPs in this exercise
is somewhat subjective. But again, this is good enough for our purposes here.
AGAIN,
SAVE YOUR WORK SO FAR!
To save your work thus far, go to File-Save. As indicated above, this
will create a file with a .psx suffix. If you wanted to quit at this point, you
could come back and open this .psx file and you would come right back to where
you left off. You should get in the habit of saving your work periodically as
you move through this project.
Input
Marker Coordinates
Follow the tutorial instructions to Input Marker Coordinates ( on the Reference pane) using
the file post_pot5gcp.txt. Use these settings:
Note that you need to specify the coordinate system
that you are using (WGS84 in our case) and make sure that you specify the
correct columns for each parameter and the starting row number, then click OK.
In the middle left panel, use the slider to see the
Error associated with each marker. At this point, it will just show the error
in terms of # of pixels; probably around 2 pixels for each one. Not great but
OK for now.
Optimize
Camera Alignment
Carefully follow the tutorial instructions here. Use
these settings:
As indicated in the instruction, be sure to uncheck
all of the images and check all of the markers, then go to Tools-Optimize
camera alignment and use:
When complete, take a look at the Error for each of
your markers. Now much better. Mine range from 0.03-0.13 m for mine. Yours will
be a bit different depending on how carefully you placed your markers.
AGAIN,
SAVE YOUR WORK SO FAR!
Set
the Bounding Box
Create
Dense Point Cloud
This is by far the most time consuming step. I’d go
with these options:
With this dataset, it may take 15 minutes or so, maybe
a bit more. I’ve had projects with 3-4000 images that I run overnight on my
computer. If you take a look at the performance tab on your Task Manager, you
will note that the CPU is maxed out during this operation. This is a good time
to take a break.
Mission
Planner: While this is running, we may spend some time
providing a brief intro to the use of Mission Planner.
After this step completes…….
Let’s take a look at the dense point cloud. To do so,
click on the dense point cloud button along the top of the screen. This should
result in a display like this:
This is analogous to a First Return LIDAR surface.
Note that this dense point cloud consists of over 7 million points. Pretty cool
isn’t it!
Drape
the image over the point cloud: click on this button to get:
And moving on…..
Build
Mesh: Do this.
Edit
Geometry: SKIP THIS STEP
Build
Texture: SKIP THIS STEP
Build
DEM: Do this. Very fast. It will looks something like this:
Build
Orthomosaic: Do this. A few minutes. It will look
something like this:
The ortho will be a cleaner image than the image
draped over the dense point cloud.
Herons?
And what about those heron nests?
Following the instructions in the tutorial, take a
look at the ortho image. Zoom in on the area of deciduous trees just to the
west of point 4 and south of the big round thing with the spokes at the sewage
treatment plant. See this:
The arrows point to a couple of Great Blue Heron
nests. Look around and see how many you can see.
Export
Orthomosaic: Do this. And note that, at this step (and
for the DEM as well), if you’d like, you can choose to export the Ortho in a
projection other than the one that you have used thus far. So, if your ultimate
goal is to use this product with some other GIS layers, you may want to export
the ortho in the same projection as your other layers. At the same time, you
can choose to alter the pixel size somewhat. For example, in this case, we’ve
been working in WGS84 with our X and Y coordinates in degrees. So the default
pixel size will also be in degrees. Like this:
Instead, you might choose to export this using a UTM
projection or State Plane with the pixel size in meters or feet. And, if you do
choose to go with UTM, for example, you will note that the default pixel size
is a bit goofy; not a nice round number. So you might choose to round it off to
something simple. Perhaps 0.04 m or 0.03 m rather than 0.0331902 m.
Export
DEM: Do this. Note that, as a rule of thumb, the pixel size
for the DEM will, by default, be about 4X the pixel size of the ortho.
If you’d like, you can bring the tiffs for both your
DEM and your ortho into Arc to play with them a bit more. This is optional.
Generate
Report:
This is not mentioned in the tutorial but I feel this
is a nice thing to do. Similar to what you did in exporting the Ortho and the
DEM, go to File-Generate Report , then
select a file name. This generates a nice PDF file with all sorts of geeky
stats on your project.
AGAIN,
SAVE YOUR WORK!
Lab
Report: OK
I’d like you to write a brief lab report based on your work. On the Lab Exercises
Index page, there is a link to a Lab Report Guide that explains the
format that I would like you to use for your report. Your report will be due on April 27 at 1:00PM.
Note
added 10/112022: What should go in your lab report? The motivation for this flight was to see if
we could use the UAS imagery to inventory (count) the Great Blue Heron nests.
I’d say the answer to this is yes. So in your results, tell me how many nests
you can find. Also note that the study area includes both conifers and
hardwoods. What is the range of heights that you find in the area with the
hardwoods and what is the range of heights for the conifers. You can get this
by simply using your cursor to sample 10-20 trees in each area and give me the
range. And give me these heights relative to the takeoff location which is at
marker location #3. You should also report your spatial resolution; the pixel
size. This is in the report that we generated from Agisoft. You might include
some screenshots from this report. What was the Error for your markers? How was
the coverage in terms of # pictures covering various parts of the study area?
In your discussion, you should also discuss what could be done to obtain better
coverage of the study area? Maybe fly lower for higher spatial resolution. But
what are the limits to doing this (height of tallest trees). Does the flight
plan provide adequate coverage of the area or does it need to be bigger? What
about our markers (Ground Control Points)? Did we have enough? Were they well
marked in the field and easy to precisely locate in the images? Did we have
accurate coordinates for these GCPs?