Geoparsing text using the geoparser.io API

Maëlle Salmon

2019-07-26

What is geoparsing?

According to Wikipedia, geoparsing is the process of converting free-text descriptions of places (such as “Springfield”) into unambiguous geographic identifiers (such as lat-lon coordinates). A geoparser is a tool that helps in this process. Geoparsing goes beyond geocoding in that, rather than analyzing structured location references like mailing addresses and numerical coordinates, geoparsing handles ambiguous place names in unstructured text.

Geoparser.io works best on complete sentences in English. If you have a very short text, such as a partial address like “Auckland New Zealand,” you probably want to use a geocoder tool instead of a geoparser. In R, you can use the opencage package for geocoding!

How to use the package

You need to input a text whose size is less than 8KB.

library("geoparser")
output <- geoparser_q("I was born in Vannes and I live in Barcelona")

The output is list of 2 data.frames (dply::tbl_dfs). The first one is called properties and contains

output$properties

The second data.frame contains the results and is called results:

knitr::kable(output$results)

You can input a vector of characters since the function is vectorized. This is the case where the MD5 hash of each text can be useful for further analysis.

library("geoparser")
output_v <- geoparser_q(text_input = c("I was born in Vannes but I live in Barcelona.",
"France is the most beautiful place in the world.", "No place here."))
knitr::kable(output_v$results)
knitr::kable(output_v$properties)

How does it work?

The API uses the Geonames.org gazetteer data. Geoparser.io uses a variety of named entity recognition tools to extract location names from the raw text input, and then applies a proprietary disambiguation algorithm to resolve location names to specific gazetteer records.

What happens if the same place occurs several times in the text?

If the input text contains several times the same placename, there is one line for each repetition, the only difference between lines being the values of reference1 and reference2.

output2 <- geoparser_q("I like Paris and Paris and Paris and yeah it is in France!")
knitr::kable(output2$results)

What happens if there are no results for the text?

In this case the results table is empty.

output_nothing <- geoparser_q("No placename can be found.")
output_nothing$results

How well does it work?

The API team has tested the API un-scientifically and noticed a performance similar to other existing geoparsing tools. A scientific evaluation is under way. The public Geoparser.io API works best with professionally-written, professionally-edited news articles, but for Enterprise customers the API team says that it can be tuned/tweaked for other kinds of input (e.g., social media).

Let’s look at this example:

output3 <- geoparser_q("I live in Hyderabad, India. My mother would prefer living in Hyderabad near Islamabad!")
knitr::kable(output3$results)

Geoparser.io typically assumes two mentions of the same name appearing so closely together in the same input text refer to the same place. So, because it saw “Hyderabad” (India) in the first sentence, it assumes “Hyderabad” in the second sentence refers to the same city. Also, “Islamabad” is an alternate name for Chittagong, which has a higher population than Islamabad (Pakistan) and is closer to Hyderabad (India).

Here is another example with a longer text.

text <- "Aliwagwag is situated in the Eastern Mindanao Biodiversity \
Corridor which contains one of the largest remaining blocks of tropical lowland \
rainforest in the Philippines. It covers an area of 10,491.33 hectares (25,924.6 \
acres) and a buffer zone of 420.6 hectares (1,039 acres) in the hydrologically \
rich mountainous interior of the municipalities of Cateel and Boston in Davao \
Oriental as well as a portion of the municipality of Compostela in Compostela \
Valley. It is also home to the tallest trees in the Philippines, the Philippine \
rosewood, known locally as toog. In the waters of the upper Cateel River, a rare \
species of fish can be found called sawugnun by locals which is harvested as a \
delicacy." 

output4 <- geoparser_q(text)
knitr::kable(output4$results)

What can I do with the results?

You might want to map them using leaflet or ggmap or anything you like. The API website provides suggestions of use for inspiration.

Meta