Sign in

Missing males in the U.S from 1955 to 2015. (Data source: The Doe Network: International Center For Unidentified & Missing Persons)

There are tons of useful data on websites we browse every day, but each site might have its own structure and form that making it difficult to extract the data you need quickly. In this article, I will show you how I gathered the missing persons' data from a website (International Center For Unidentified & Missing Persons) by using Scrapy, and how I cleaned the data and analyzed the spatial-temporal data with ArcGIS tools.

The webpage of the missing males from The Doe Network: International Center For Unidentified & Missing Persons http://www.doenetwork.org/mp-chrono-us-males.php

Scraping the website

Scrapy is…

Xinyue Xu

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store