Published on GEOG 865: Cloud GIS (https://www.e-education.psu.edu/geog865)

Home > Lessons > Lesson 6: Thematic mapping services with CARTO

Lesson 6: Thematic mapping services with CARTO

Overview

There are many interesting offerings of cloud GIS SaaS. This week, we'll try CARTO [1] (formerly CartoDB). Worth noting about CARTO is that it is an open-source project. You could completely replicate what they have, using your own Linux server. At the same time, CARTO is able to operate a business by selling their services to the many folks who would rather focus on simple mapping on the cloud instead of deploying the entire software themselves on their own hardware.

The source code is available at the GitHub website [2]. They are using a fantastic set of technologies, although it might be quite a job keeping up with all the dependent projects if you wanted to work on the source. Fortunately, there's no need, as we can use their free pricing tier to get a feel for their cloud offerings.

Lesson Objectives

At the successful completion of this lesson you should be able to:

  • understand CARTO software and how it enables thematic mapping;
  • analyze spatial data patterns using CARTO aggregation tools;
  • visualize complex data in CARTO using time-series animations; and
  • understand how to upload datasets to CARTO.

Deliverables

  • Complete: L06: Assignment
  • Participate: L06: Discussion

CARTO mapping and location services

CartoDB was officially launched in 2012 as a web mapping front end to a PostgreSQL + PostGIS back end database. The software was open source and could run on one's own hardware, but at the same time CartoDB offered an online subscription service wherein customers could upload datasets and make maps without having to touch the source code or configure anything themselves. In 2015, one of the best-known PostGIS masterminds, Paul Ramsey, joined CartoDB [3]. In 2016, the company changed its name to CARTO [4] and repositioned itself as a "location intelligence" tool rather than just a basic web mapping interface and online database. As such, it now also offers geodemographic analysis, routing, proximity, and address finding services.

Some of the services, you will note, are similar to those offered by the other SaaS offerings we are studying in this course: Mapbox and ArcGIS Online. This is inevitable because these companies have have found an eager market for the kinds of services they offer, and competition is a byproduct. In these course lessons, we have tried to focus the walkthroughs on some of the unique strengths of each platform or the technologies that they pioneered. One of CARTO's unique points is its variety of thematic mapping options and its appealing basemap and thematic styling options. The color schemes use the ColorBrewer [5] ramps which were developed at Penn State and are based on scientific color theory. Cartographers using CARTO can aggregate point data to tesselated regions such as hexbins or their own boundary files that they upload. They can also make time series maps, rasterized heatmap-style density surfaces, proportional symbol maps, etc.

CARTO offers a "Builder" app for web-based design and an "Engine" piece consisting of APIs. CARTO services can perhaps be considered as either PaaS or SaaS. How can we distinguish between them? One way is to consider how they will be used. If the service is being used as a source and combined with others, then it is probably a platform service. If it is being consumed directly by the end user, then it's a software service. Along the same lines, if you access the service programmatically, it's more likely to be a platform service than if you access it with a GUI.

So, if we used CARTO as a source of web maps that we pass along to end users, then it's a software service. If we use CARTO as a "table in the cloud" then we would be using it as a platform. CARTO's provision of spatial data tables on the Internet, along with both GUI and programmatic access for users and programmers, makes them a good example of a cloud GIS.

Uploading data to CARTO and aggregating to hexbins

Let's go through the steps of uploading a basic dataset to CARTO and making a web map.

First, download the data for this lesson [6]. This folder contains datasets that I derived from Portland Maps Open Data [7] and OpenDataPhilly [8]. They are stored in GeoJSON, a popular format based on JavaScript syntax that is used for interchanging vector data on the web.

Then follow these steps:

  1. To log into Carto, in a web browser, visit carto.com
    1. Click the Try For Free button to create a 14-day trial account.
    2. Login to the Carto Platform through the login page [9].
    3. If you are promtped to create an Organization, create a new unique one with your name or Penn State Access ID.

      (There is an option to create a free student account via GitHub, but I have had almost no success getting that to work in previous semesters due to GitHub's verification process. They seem to have issues with us being in an online program and not physically at University Park, PA.)
       
  2. After you log in, you will see options along the left of the CARTO homepage including Maps and Data Explorer. We'll use the Data Explorer link to upload and view our datasets, and the Maps link to design some maps from them.
  3. Click Data Explorer and click the Import Data button.
    1. Browse to csa_farm_dropoffs.geojson on your computer.
    2. Uncheck the Auto-Guessing slider.
    3. Upload the data.

      You should now see the dataset in the Data Explorer list. These are locations where local farms around Portland will bring their produce into the city. They drop it off at these points for urban residents who have signed up for community-supported agriculture (CSA) subscriptions. Mapping these points is one way to see which areas of Portland are currently being served by CSAs (and which ones are not).
       
  4. In the main menu along the left, enter the Map section and click the button to Create Your First Map.
  5. In the map window, use the Add Source From button to add your farm dataset to the map. You'll see the points overlayed on a basemap of Portland. You're now in CARTO Builder, which is a map design center similar to the ArcGIS.com map viewer or Mapbox Studio (although Builder is more for styling thematic maps rather than basemaps).
  6. At the very top of the window, give your map a name, and notice the Updated indicator beside it. Your map will be saved automatically as you work.
  7. Click the Basemap link and explore the variety of basemaps offered by CARTO. For example, try the Dark Matter layer. These are built from OpenStreetMap data. You can choose your favorite one for use in this exercise.
  8. Go back to the layer list, and click on the farm points layer.
  9. Play around with changing the fill, outline, and other properties of the points.

    Sometimes it's easier to visually make sense of a bunch of dense points by aggregating them to polygons of uniform size. Hexagons are a popular choice because they are compact and tessellate (fit together) easily. Let's aggregate these points to hexagons, or "hexbins" as cartographers sometimes call them.
     
  10. Still viewing the properties of the csa_farm_dropoffs layer, choose the Hexbin option, and select a color ramp and size for the hexbins that appeals to you. This is somewhat of an art; if the hexbins are too small, you won't learn much more than if you viewed the raw points, and if the hexbins are too large, the patterns in the data will be muted.
  11. Click the color ramp, and notice the options to change the number of classes that the data is divided into. You can also change the classification scheme. Right now it's set at Quantile, which ensures that there will be an equal number of hexbins in each class, or color.
  12. Set a classification scheme and number of buckets that you feel best tells the story about where Portland is serviced by CSAs. Notice how much power you have over the message implicit in the map patterns. This is true with all cartography and not just online tools, yet interactive web GIS of this nature brings the power of the cartographer to the forefront.
  13. Use the Share button at the top of the window to Publish your map and generate a Public Share Link. Paste that url link into a new browser window to see how your map may be viewed by other users. You'll also include this link as part of your assignment submission.

Making an animated time-series map

Let's try one more kind of map that CARTO does very well: the animated time series map. This type of map is used when your data has a date and/or time field representing when an event occurred. The data we'll use represent incidents of gun violence in Philadelphia. Each point is a shooting with a field noting when the event took place. Animating these events over time within a map can show temporal and spatial patterns of violence throughout the city.

  1. Following the techniques you learned in previous lessons, upload shootings.geojson to CARTO, this time leaving the Auto-guessing slider on. This way, the data types will be preserved, in particular, the Data field we'll need for the time-series widget.
  2. Look at the table in the Data Preview section of Data Explorer.
  3. Note the different fields that contain a date and/or time. Which one do you think represents the date the shooting occurred? If you said date, you're right. What we don't want to animate is the date that the point was added or updated within the GIS itself (unless we have no better temporal information), therefore we'll ignore the created_at and updated_at fields.
  4. Create a new map with the shootings data.
  5. Use the tools in the CARTO Builder to add a Time Series widget using the shootings data you uploaded. 
  6. Be sure that the field the widget is using is the Date field.
  7. Be sure that Animation Controls slider is on.
  8. Click on the widget and look at the panel below the map. You should see a Play button that will animate the map, showing events over time.
  9. You can continue to apply other thematic styling to the layer beyond the animation. Let's change the officer-involved shootings to a red color and leave all the other ones blue.
  10. Use the layer style tools to creat a custom color scheme with two colors: red and blue, and specify that it is an Ordinal Scale. Next to the Point Color, click BY VALUE, and choose officer_involved. Set a value of Y to red and N to blue. Now you should be seeing the occasional red dot appear among the animated incidents. You may want to increase the size of the point symbols to see them more easily.
    The animation may be playing a bit fast for your taste at this point. There are lots of settings you can adjust to determine how fast the dots appear and how long they stay on the screen.
  11. Use the Share button at the top of the window to Publish your map and generate a Public Share Link. Paste that url link into a new browser window to see how your map may be viewed by other users. Explore the functionality, in particular, the ability to click and draw a range of times in the animaion window to aggregate activity temporally. You'll include the link to your animation map as part of your lesson deliverables.

Assignment: Doing your own mapping with CARTO services

In this week's assignment, you'll continue getting some experience with CARTO's thematic mapping services. Please assemble a document with all of the following:

  1. A hyperlink to your CSA map aggregated to Hexbins.
  2. A hyperlink to your time-series map of Philadelphia shootings.
  3. A map of one of your own datasets that appropriately applies either the aggregation or time series techniques covered in the walkthroughs. Write a paragraph explaining what you are showing and justifying your design decisions regarding styling, classification, etc.
  4. A map of your own datasets made using some new technique you've learned by experimenting in CARTO Builder and reading its documentation. Add a paragraph explaining what technique you are using and why it's appropriate for showing these datasets.

    Items 3 and 4 can use data from your final project or some other data that you've obtained from an open data website. Please be sensitive to data ownership and licensing when uploading data to CARTO; don't put your organization's proprietary data on CARTO unless you've received permission to do so.

Cloud Computing Discussion: Reliability at Cloud Scale

This week's cloud computing discussion covers Service Oriented Architecture (SOA) and Hadoop-style massively parallel data processing systems. SOA is interesting because this is how new Internet services are being developed. It is also a huge engineering challenge.

An epic blog post that helped me understand the importance of this was written by software engineer Steve Yegge [10], it is known as Stevey's Google Platforms Rant [11]. Yegge used to work for Amazon and now works for Google. Apparently, it was meant to be internal to Google, but it was accidentally published to great acclaim. Please read it for his passionate advocacy of a service oriented architecture and developer tools, and for his rather humorous, if somewhat salty and irreverent, description of life while working at these software companies.

Hadoop is an amazing system that was started by Doug Cutting, who wanted to provide the means to be able to index the entire Internet overnight, which at the time, only Google was doing effectively. Please read the Wikipedia entry on Apache Hadoop [12] for background. Hadoop is quite powerful, but also notoriously tricky to get working. Amazon has an interesting service called Elastic MapReduce [13] which claims to take a way a lot of the pain of setting up and maintaining such systems.

Deliverables for this week's cloud discussion:

  1. Read Stevey's Google Platforms Rant [11], the Wikipedia entry for Hadoop [12], and the introduction to these topics earlier in this page.
  2. Post a comment in the lesson discussion in Canvas that describes how SOAs and parallel data processing systems, as described in these articles, could benefit GIS applications and services.
  3. Then I'd like you to offer additional insight, critique, a counter-example, or something else constructive in response to one of your colleagues' posts.
  4. Brownie points for linking to other technology demos, pictures, blog posts, etc., that you've found to enrich your posts so that we may all benefit.
  5. If there are concepts or vocabulary items that are not familar to you, don't suffer alone! Please post a question in the Technical Discussion Forum in Canvas. Posting a question is a form of participation, but doesn't take the place of your substantive post requested in step 2 above.


Source URL:https://www.e-education.psu.edu/geog865/node/35

Links
[1] http://www.carto.com [2] https://github.com/CartoDB/cartodb [3] https://carto.com/blog/paul-ramsey [4] http://cartodb.pr.co/131800-cartodb-launches-a-self-service-location-intelligence-tool-and-rebrands-itself-to-carto [5] http://colorbrewer2.org [6] https://www.e-education.psu.edu/geog865/sites/www.e-education.psu.edu.geog865/files/data/carto_lesson_data.zip [7] http://gis-pdx.opendata.arcgis.com/ [8] https://www.opendataphilly.org/ [9] https://carto.com/signin/ [10] http://en.wikipedia.org/wiki/Steve_Yegge [11] https://gist.github.com/1281611 [12] http://en.wikipedia.org/wiki/Apache_Hadoop [13] http://aws.amazon.com/elasticmapreduce/