Automated detection and classification of scallops, flounder, and habitat
 

Project leads: Liese Siemann and Tasha O'Hara

Collaborator: Matt Dawkins, Kitware, Inc.

Funded by: NOAA Sea Scallop Research Set-Aside

CFF began collaborating with Kitware in 2019. The objectives of our first project were to develop automated detectors for different classes of scallops (live vs. swimming scallops vs. clappers) and flounder using the stereo image pairs taken during our HabCam surveys. Our current project is focused on automating scallop size classification and developing detectors to classify benthic habitats. Data collected during this project will be used to study habitat preferences of juvenile scallops. Our projects contribute to a broader NOAA effort to support development of Video and Image Analytics for Marine Environments (VIAME), an open-source system for analysis of underwater imagery that is being developed by Kitware. Our partners in this effort are early supporters of VIAME, including scallop assessment scientists at the Northeast Fisheries Science Center.

Human annotation of images is the most time consuming part of generating reliable biomass estimates from optical surveys, and humans can be imprecise, particularly when classifying substrate types. and substrate classification. To develop accurate automated detectors, CFF has provided Kitware with annotated imagery from our 2017-2020 surveys. We are currently developing a classification scheme based on the Coastal and Marine Ecological Classification Standard to accurately describe habitat types.

VIAME interface with scallop and flounder.jpg
3D stereo swimming scallop figure.jpg

To accurately measure animals that could be resting or swimming off bottom, determining their height off the seafloor is critical. Accomplishing this starts with generating depth maps from stereo images. These depth maps can also be used to create 3D renderings of the sea floor.

dense scallop annotations.jpg

Annotating dense scallop beds is a time-consuming process.

flounder annotations.jpg

Automated detectors can find scallops missed by human annotators.